Rearchitecturing the backend evaluation

[Almenon]

Is your feature request related to a problem? Please describe.
There are a number of problems that can appear when doing the second run of AREPL. For example, #438 and #436. I've tried adding code in my backend to address these issues, but it adds complexity to the backend and does not fully solve all issues.

Describe the solution you'd like
A better solution would be a double-process approach. When the extension starts, two processes are spawned, A and B. When the debounce finishes, code is passed to A. When A returns, A is restarted. To prevent code waiting on A restart, when A is running or restarting, code is passed to B. The backend starts fast, so by the time the next input rolls around, A will be ready to go. It will go to A, then B, then A, and so on.

This means I can get rid of a lot of backend logic, because each process will start entirely fresh each time. It will also completely eliminate any bugs or cases not handled by said backend logic.

Also note that previously AREPL would throw away input that came in while AREPL was executing. Now AREPL will always show the results from the last input, which is a plus.

Scenarios:

A executing with incoming code -> A force-restarted, B executes incoming code. 
A restarting with incoming code -> B executes incoming code. 
A online with incoming code -> A executes code

A executing with incoming code & B restarting -> A force-restarted, B executes incoming code when restart completes.

A and B both executing -> impossible, outside some ultra-horrible race condition. 
A and B both restarting -> Might be possible if the restart is delayed enough. Code should wait for restart to finish.
A and B both online -> This is fine.

Result is received from A -> A restarts
Result is received from B -> B restarts

Might be able to simplify above logic if I just toss enough executors in there that there will always be one available. Downside is that uses up more memory (40MB each process).

Describe alternatives you've considered
The simplest solution would be to just restart the process every run. However, I tried it and it adds slowness to the time you wait for execution. This slowness might be acceptable, but in the absence of user feedback I'm going with my personal feeling, which is unfavorable.

Additional context
numpy/numpy#16241

[Almenon]

This will also fix #367

[Almenon]

There would be two or more processes. This would be the lifecycle for each process:

flowchart TD
    P[process created] --> S[Starting] --> |Start result received| F[Free] -->|incoming code| E[Executing] -->|Result Received OR incoming code| S

Loading

Not necessarily in order:

1. When a process is created, the process is set as Starting. This status persists until it receives a special start result which indicates that the process has finished starting.
2. AREPL would keep track of the free processes. When code comes in, it would restart all executing processes (we don't care about the result of old code). It would then wait for a free executor to exist. When it does, it would set the first free executor as executing and then send it the code.
3. When the result is received, it would mark the executor as starting and then restart it.

Note that restarts are not immediate. AREPL backend gives a slight delay before force killing to let any end handlers finish up.

Another diagram. In this scenario three processes start, code comes in, code finishes, code comes in, and then another code comes in while the first code is executing:

processes | time
SSS           1
FSS           2
ESS           3
EFF           4
EFF           5
EFF           6
EFF           7
SFF           8
FFF           9
EFF           10
SEF           11
FFF           12

[Almenon]

So that's the theory, but how to implement the code?

I could set a setInterval, checking the executors every X milliseconds till one of them are free. New code would clear the setinterval and set a new one.

Or maybe I could use https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/any.... somehow? The problem is I'm dealing with several different functions. the handleresult function is different than the onUserInput function.

I suppose I could refactor the backend to return a promise when execCode is started, and store that promise somewhere. Then, something like:

if(freeProcesses){
  exec()
}
else{
  Promise.any([startingPromise, executingPromise]).then(value -> value.exec(code))
}

First solution seems simpler.

[Almenon]

This reminds me of the elevator problem.

https://leetcode.com/discuss/interview-question/object-oriented-design/124936/design-an-elevator-system
https://leetcode.com/discuss/interview-question/object-oriented-design/124927/write-elevator-program-using-event-driven-programming/123400
http://play.elevatorsaga.com/

Also reminds me of load balancers, although that's actually simpler because you don't have to cancel existing processing when another request comes in.

[ben-pelletier]

Any way this can be manually worked around for the time being?

[Almenon]

@ben-pelletier which issue are you experiencing?