Implement Platform, Process, Cmd and Sub from Elm 0.18

[rgrempel]

This should be fun!

Once finished, this will enable "headless" Elm programs, and will essentially completely implement the elm-lang/core from Elm 0.18. The next step would then be to implement elm-lang/html.

[rgrempel]

Well, this is going nicely.

I've now got the "surface" API of the effects managers roughed in, so that you can translate and type-check the "visible" Elm code in effects modules. All that's left is the hard part, actually making it do something.

I'm debating whether to try to "mimic" what Elm actually does in its run-time, on the one hand, or simply implement what the types would require, on the other. I'll probably end up doing a bit of both.

In any event, I'll be caught up in other things for a while now, so I'm not sure how soon I'll get back to this.

Along the way, I switched the Task implementation to be based on purescript-io, which is a wrapper over purescript-aff that eliminates the effects rows. This turns out to be very nice for purescript-elm, since Elm's Task does not have effects rows, so it's easier to mimic the Elm API more closely.

[rgrempel]

I've taken a quick look at what's in Elm's Native/Platform.js, and it's actually more regularly structured than I had anticipated -- you could imagine doing a translation of that to Purescript. However, I'm still making progress using the "follow the types" approach, so I think I'll keep doing that until I hit a dead end.

One thing that is neat in Platform.js is the way in which ports get implemented as a special kind of effect manager ... I will need to keep that in mind when I try to implement ports!

[rgrempel]

Well, all that's left is

• runProgram
• sendToApp
• sendToSelf

Everything else is in place ... it's quite neat, actually, how far you can get by just following the types.

[rgrempel]

So, what I'm working on now is essentially the initiialize function from Platform.js, so here:

https://github.com/elm-lang/core/blob/b06aa4421f9016c820576eb6a38174b6137fe052/src/Native/Platform.js#L98-L139

Parts of it are straightforward enough, but it becomes apparent that initialize is using a ProcessId in a manner that is a little bit more sophisticated than the exposed Elm API suggests. (And perhaps Task as well, but we'll see).

So, thinking out loud a bit here, you create a ProcessId via Task.spawn. Here's what you get from that:

function rawSpawn(task)
{
	var process = {
		ctor: '_Process',
		id: _elm_lang$core$Native_Utils.guid(),
		root: task,
		stack: null,
		mailbox: []
	};

	enqueue(process);

	return process;
}

I'm not paying too much attention to the enqueue for the moment, though it may end up being important. (It's Elm's way of getting the process to be executed, which Aff will also manage, though presumably in a somewhat different manner -- I may need to look at the details).

What's interesting is the mailbox ... it implies, as it turns out is correct, that you can communicate with a ProcessId. This isn't in the official Elm API yet ... the only thing you can officially do with a Process in the Elm API is kill it.

So, how does one make use of this mailbox? Here's the send function:

function rawSend(process, msg)
{
	process.mailbox.push(msg);
	enqueue(process);
}

function send(process, msg)
{
	return nativeBinding(function(callback) {
		rawSend(process, msg);
		callback(succeed(_elm_lang$core$Native_Utils.Tuple0));
	});
}

So, this would have an Elm function signature roughly like:

send :: Process msg -> msg -> Task x ()

Now, of course the Elm process is not parameterized, but this kind of process would ideally need to be. Come to think of it, the "future directions" comment in Plstform.elm suggests as much -- it speculates that the future API for a ProcessId might look like this:

type Id exit msg
spawn : Task exit a -> Task x (Id exit Never)
kill : Id exit msg -> Task x ()
send : Id exit msg -> msg -> Task x ()

But, in fact, it seems that it already is this way under the hood, at least to a degree.

Just as an aside, this ability to send messages between processes is just a bit reminiscent of Elm 0.16's signals ... in a sense, this might be where the idea of signals pokes up its head in Elm 0.17. Also, it has an interesting relationship to effects managers, which I'l need to explore.

So, send would add a msg to a processes mailbox, and then enqueues the process ... this lines it up to be fed to step. Now, what will step do with it? Essentially, each process maintains a stack of the tasks to be done on it ... roughly a Javascript implementation of a free monad concept. So, the step function basically does the next task, whatever it is.

Then, there appears to be a task which would "receive" a message on the processes's mailbox, if there is a message:

		if (ctor === '_Task_receive')
		{
			var mailbox = process.mailbox;
			if (mailbox.length === 0)
			{
				break;
			}

			process.root = process.root.callback(mailbox.shift());
			++numSteps;
			continue;
		}

So, how do you get one of those tasks? There is a receive function to construct one:

function receive(callback)
{
	return {
		ctor: '_Task_receive',
		callback: callback
	};
}

Now, what's the Elmish signature of the receive function? I think it's something roughly like

receive : msg -> Task ? ?

So, it doesn't do any good to send a message to just any process ... it has to be a process which will, eventually, receive them. (That is, the task you turn into a process must, when it wishes, explicitly call receive in order to pick up the message). This is set up in Platform.js by a kind of recursive task that calls receive, processes the message, and then calls receive again (forever, so to speak).

I doubt that Aff's Fiber has anything quite like this, though I'll look. In Purescript terms, this might have something to do with continuations, or with streams -- ideally, it will be another monad transformer that I can layer on top of ExceptT and IO (which is what Task is currently composed off). Anyway, I'll need to take a look.

The other alternative would be to make this kind of Process a special kind of process, but I'd prefer to avoid that -- it would be far more interesting to figure out what this actually is in Purescript terms.

[rgrempel]

Ah, Aff's concept of an AVar, takeVar and putVar may be relevant here ... it looks as though that could be used to implement the idea of a mailbox ... each process, when created, could get an empty mailbox, and then there would be functions analogous to send and receive that could be made to do the right thing. It might be no more complex than that.

[rgrempel]

I've been working away at implementing a kind of send and receive for processes and tasks. Roughly, the plan was:

• spawn would create an AVar msg mailbox for each Process ... so, the Process would have both a fiber and a mailbox

• send is then easy ... it's just a putVar to the mailbox for the process

• conjuring up a Task for receive is a little harder, but basically you can add a ReaderT (AVar msg) to the monad stack for Task, and then you'd only be able to run a task inside a process ... that is, only after being spawned, at which point you've got your mailbox, so you can do a runReaderT. So, you'd end up running processes, rather than tasks.

Working away at this for a bit, it was reasonably clear that it could be made to work. However, it meant that ProcessId would need a type parameter msg ... that's basically inevitable, since obviously you need to track what kinds of msg a particular process can handle. It also meant that Task needed an additional type parameter for msg, at least if receive was going to be an ordinary Task. We could, after all, only receive in a Task the kind of the messages that the task's process would ultimately allow (given a single mailbox per process).

And that would be doable, of course, but I'm trying to avoid adding type parameters where Elm doesn't have them. That is, I'd rather not have ProcessId msg where Elm just has ProcessId, or Task msg x a where Elm just has Task x a. Now, there are ways to deal with that, via type aliases, existential types, etc., which sometimes works nicely, but is sometimes a real pain.

Thinking that over, it occurred to me that, at least for now, this plan was actually more ambitious than it needed to be. The plan would have allowed constructing tasks and processes that at arbitrary moments waited for a message to be sent to the mailbox, and then went along and did other things. That's probably something that Elm will have eventually, but it's not what it has now -- the only thing which actually sends or receives messages now is the kernel itself. And, it does so in what is essentially just a very tight loop -- it just waits for a message, then dispatches it in a characteristic way.

So, I don't really need (yet) to integrate the sending and receiving of messages between processes into the ordinary flow of tasks. Instead, it can be a "special" thing, to implement the specific things that the kernel does.

Now, of course, the Elm kernel uses tasks to implement all of this, but that's mostly because what it's got is tasks. In my case, I don't necessarily have to bring this into Task ... I can use whatever works best.

So, now I'm imagining a set of distinct types to implement the stuff that Elm is doing "under the covers" here. Ultimately, it remains true that the message passing will involve AVar, of course.

[rgrempel]

So, I've reverted the commits that added a mailbox to every process, and I'll start again with a separate type for receiving & dispatching messages.

[rgrempel]

Yes, Plan B is much better ... the main event loop and the event loops for the effects managers are now mostly done. It ended up being easier not to implement the Elm Javascript too literally -- it was, once again, better to follow the types, as usual.

[rgrempel]

Eureka!