Add go keyword

[applejag]

I'm mostly rewriting my Bash scripts in Risor.

One feature I'm really missing is the "run in background" (bash: &, go: goroutines)

Adding this would not be super easy feat, as it requires new thinking in the vm. Should it be proper multi-threading, or can some simpler solution be sufficient?

To not have to deal with race conditions, I suggest doing an event-based solution on a single thread. Similar to languages like JavaScript and Lua.

As for syntax, as Risor pivots itself towards Go developers, I think it'd be only natural if it was using the go syntax.

However, instead of having to deal with mutexes, wait groups, channels, etc, I suggest that a go expression returns a handle to the coroutine. Similar to JavaScript's Promise, or C#'s Task.

So that code like this could be possible:

# a, b, and c are of type "coroutine"
a := go fetch("https://example.com/a")
b := go fetch("https://example.com/b")
c := go fetch("https://example.com/c")

# `.wait()` is a method on "coroutine", returning the normal result from `fetch`
a_result := a.wait()
b_result := b.wait()
c_result := c.wait()

printf("result from a: %v\n", a_result.json())
printf("result from b: %v\n", b_result.json())
printf("result from c: %v\n", c_result.json())

If the function had an error, then .wait() would also return that error.

Avoiding function coloring

The hot topic of the year in language/library design is: What Color is Your Function?

I think that taking a similar approach to Bash's "job control", Go's "goroutines" and Zig's "colorblind async/await model" would be good.

Meaning, all functions are just normally synchronous. But then you can tell the runtime to run a certain function in the background.

What happens with un-waited coroutines

Given this:

go fetch("https://example.com/a")

What should happen, is that the Risor VM should just automatically wait for any dangling coroutines at the end of execution.

In practice, it would as if it ran this:

task := go fetch("https://example.com/a")

task.wait()

But implemented in the VM, not in the compiler.

Implementation

A simple way to implement the coroutines would be as in the definition of a coroutine. It's a function that can be suspended, and later resumed.

My proposed approach:

• Risor script file is considered a new Risor coroutine
• When you hit a x := go myfunc()
  • capture the arguments and store that in the Risor coroutine object
  • add the Risor coroutine object to a queue for potential future processing
• When calling x.wait()
  • if underlying type is goroutine"
    • if it is finished, return its result
    • else, skip to next Risor coroutine
  • if underlying type is Risor function:
    • just execute the function
• When all code finished executing, then start executing the next Risor coroutine in the queue

IO operations (e.g fetch, exec.command) would need to be implemented as:

• start the operation in a new Go goroutine
• create a special Risor coroutine object who's .wait() will wait for the underlying Go goroutine to finish
• the fetch(...) function itself calls this .wait() function

This is basically an event-based multi-threading model. The IO operations make use of Go goroutines to actually run in parallel, but the Risor code is always executed on the same thread in sequence.

This is very much akin to how languages like JavaScript, Lua, and even most Rust libraries deals with coroutines, except that in this proposal it inverses the calling convention of async functions, so that they're the same as non-async functions.

So one way to look at this is that a function call would always try to "await" the function if it is "awaitable", but otherwise just run it as usual.

Drawbacks in implementation

One thing that might be considered confusing is when a piece of code is actually executed.

Consider the example:

func my_func_that_logs(message) {
  fmt.println(message)
}

// Calling "go my_func_that_logs" doesn't actually run any of its code yet
// It only captures the arguments and function
first := go my_func_that_logs("first")
second := go my_func_that_logs("second")
third := go my_func_that_logs("third")

third.wait()
first.wait()
second.wait()

The output here would be:

third
first
second

While the following:

func my_func_that_logs(message) {
  fmt.println(message)
}

go my_func_that_logs("first")
go my_func_that_logs("second")
third := go my_func_that_logs("third")

third.wait() // run now

// end of script, so start executing dangling coroutines

Outputs:

third
first
second

Alternative implementation

Same as before, but when entering a go myfunc(), start executing as much as possible, until you reach a suspending IO operation.

This is akin to C#'s multi-threading implementation.

And in effect then the above example would be:

func my_func_that_logs(message) {
  fmt.println(message)
}

go my_func_that_logs("first") // run now
go my_func_that_logs("second") // run now
third := go my_func_that_logs("third") // run now

third.wait() // task already completed, so effectively a no-op

Outputs:

first
second
third

Scope of changes

• The Risor VM needs to handle suspending and resuming coroutines, by juggling call stacks and instruction arrays. Each coroutine could be considered its own VM, and the scheduler just decides which one is the current one.
• The parser stack needs to handle the new go keyword.
• The compiler would translate the go keyword into the equivalent of just a function call, like go myfunc(1,2,3) = go(myfunc, 1, 2, 3). No complex data shuffling needed here.
• The standard library/builtin modules would have to be updated to make IO functions make correct use of Go goroutines and Risor coroutines.

[myzie]

Thanks for writing up this proposal. At a high-level, I agree that making a goroutine equivalent in Risor is needed.

You kind of touched on this I think, but I have thought that something like the Javascript Web Worker API might do it. Strict isolation, with no data sharing, only messaging.

Having each coroutine actually have its own VM could be part of the solution, potentially. You also mentioned this in passing it looks like.

Sorry I'll spend more time reading your proposal in detail soon.

[applejag]

Oh yeah! Sparks the ideas!

Maybe interesting to just lean on Go's concurrency model as much as possible. Where go my_func() (Risor code) literally translates to just go run() (Go code) in the Risor VM.

Maybe a different syntax is needed too then. And the different Risor VMs would not be allowed to access variables from other Risor VMs, to prevent race conditions.

In practice, that would mean no captures in Risor coroutines, which might be a big blow. So the API usage in Risor needs to reflect that in a nice way so it doesn't feel super constraining.

JavaScript Web Workers get around this by forcing you to use separate files, something that I don't think is a good fit for simple Risor scripting.

[luisdavim]

What do you think about this approach? #155

[applejag]

The go keyword was added in #178