Transient and Unison

(From https://gitter.im/Transient-Transient-Universe-HPlay/Lobby?at=5d0dea29d1aaa16964e9f3e3)

Unison: https://github.com/unisonweb/unison

Unison install procedures remotely and execute remote procedure calls (RPC), while transient install whole programs and transport closures. the difference is:

      -- in transient
          x <- localProc
          y <- localProc'
          z <- runAt remoteNode  $ do
                       process with all the variables
                       instantiated in the  closure: x, y ....
                                             

     -- in unison
         x <- localProc
         y <- localProc'
         z <- runAt remoteNode remoteProc(x,y)

   remoteProc (x,y)= .... 

The former gives more freedom and composability.

This can not be done in RPC

        x <- 
        z <- runAt remoteNode  $ \x z-> ....

A problem of RPC is that all remote calls should address static procedures. It is not possible to use lambdas so the above example should not work in unison, since it uses a lambda. Better said, RPC can not invoke methods with free variables.

It is quite possible that the compiler could rewrite that lambda as a static function and make it work. But there are functions defined locally in the closure that can not be rewritten that way since they have "free" variables:

proc x = do
    let   localProc y =  expression depending on x
    runAt remoteNode   localProc wathever

This localProc can not be rewritten as static, since there are one static localProcs possible for each value of x, since x is a free variable.

Anything can be overcome. Since Unison is dynamic, maybe the compiler could detect free variables and compile the appropriate static in the remote node, but this is far from easy doing it efficiently. In practice it is equivalent to transporting closures, that is what transient does.

However, the idea of Unison for managing and distributing software is amazing

Transient uses a program, the monitorService running in each node which receives request for compiling/installing other services or programs. they are identified by his name and referenced by a URL to a git repo or a docker image, or a unix executable. It has to install whole executables. The closure data transporting mechanism need whole programs to be installed, since a remote function which is localy defined in the body of another function ad so on needs the whole context to address it. Unison RPC does not need to install whole programs. It identifies particular chunks of code by a hash. They are retrieved, compiled and cached when the code execution is requested in the node.

In Transient, only the data (local variables) of the closure are transported at invocation time. The calling and the callee should share the same stack structure. In práctical terms, this means that both node should run the same program, although nodes may have different architectures (A Javascript node in a browser can communicate with the server). To communicate programs with completely different codebases, transient has an RPC-like mechanism, but this comes for free, since it uses the closure transport using something like a pseudo-closure and is also reactive as we show below:

In the other side, as far as I know, unison as any RPC implementation, is not reactive. this means that a remote node can not stream events back to the calling node using a unique interface. In transient, a remote node can return zero, one or many responses using the same invocation primitive runAt:

     event <- sendFromRemote
     processLocal event
     where
     sendFromRemote nod = runAt node $ choose [1..]   -- stream the numbers back

That is very important because most distributed applications do stream events (data bursts, messages, communications, events, name them as you like) instead of single results.

closure transport allows more possibilities: while a typical RPC call is a remote function invocation similar to runAt, in Transient that primitive is a composition of two more basic ones:

runAt node f = wormhole node $ do 
   teleport; 
   r <- f ; 
   teleport ; 
   return f 

wormhole opens a connection and teleport transport back and forth the computation between these two nodes connected.

This is key for being reactive and streaming; f by the first wormhole is running at the remote node, If f generate events, the continuation would execute the second teleport, which will transport the data to the calling node that connected through wormhole. This is the reason why transient can stream events. By default, each one of these responses run in his own thread.

Moving the whole execution state and continuing in other node is not possible using RPC. In transient it is indeed possible for a program to distribute his execution among many nodes. The f above has access to all the previous execution context that ran in the calling node. it is even possible to store the execution state and continue it at a later time. A system based on RPC can not do so "natively".

execution states are lightweight because they include only the serialization of the variables that are actually on scope (an not the "dead" ones which are no longer visible). Additionally, only the new variables are transported. In the above program, only the r value would be sent back, even if that is invoked locally in a complex context.

Since the TransIOcomputation can return zero, one or more responses and these responses can be managed by different threads, the f above could either never return anything, be conventional and return a single result, but also can send back results (events, stream of results, call it as you like) to the calling node and the programmer can decide how many thread would handle these responses.

Installing whole programs have his advantages: it can use conventional installers or package/aplications managers while Unison requires an innovative software installation system and version control which is very promising but poses very hard problems, like how to interact with conventional languages/applications operating system configurations etc.

In the other side the effect system of Unison, based on Fran and the language itself is way more friendly and has less accidental complexity than Haskell from my point of view. I wish Unison and specially the language all the best.