allow recursive/nested ticks

[eghteentwelve]

Please allow for recursive/nested ticks, i.e. functions registered with process.nextTick(). Specifically I'm talking about modifying below node.cc code to remove in_tick guards:

if (tick_info->in_tick()) {
  return ret;
}

// …

tick_info->set_in_tick(true);

env->tick_callback_function()->Call(process, 0, NULL);

tick_info->set_in_tick(false);

I'm asking this to enable correct operation of add-ons utilizing libuv nested loops. If this request cannot be done, please explain how recursive/nested ticks can happen today in user-land or, perhaps, it's only an issue in node-core?

many thanks

[othiym23]

See my explanation on the nodejs Google group for what (I understand) is going on in node.cc. What are you trying to do with libuv / your add-on that requires "nested loops"? As far as I know, nested loops aren't really a thing in libuv – you can have multiple run loops, and you can even (if you're crazy like @trevnorris) make libuv work in a multithreaded runtime, with multiple V8 isolates and everything, but each tick needs to run to completion before you start another one on a given thread. Right now, I don't think there's enough info here to say whether or not there's anything incorrect or restrictive about how Node is dealing with the event loop.

[bnoordhuis]

I'm asking this to enable correct operation of add-ons utilizing libuv nested loops.

Are you calling uv_run(b) from inside uv_run(a)? That should work but it will block the outer uv_run() until the inner uv_run() returns. That isn't necessarily a problem if you pass in UV_RUN_NOWAIT.

If you're trying to recurse - i.e. call uv_run(a) from inside uv_run(a) - that won't work. I've toyed with the idea of making the event loop re-entrant^ but there was a performance cost associated with it that I didn't think was worth it.

^ So I could use libuv with Chicken Scheme. Chicken is strictly CPS, functions never return. If your native code calls a scheme function, it's going to be CPS as well, whether it likes it or not. Ergo, the need for a re-entrant libuv.

[eghteentwelve]

i'm hoping to call uv_run(b) form inside uv_run(a), but a and b are not mutually exclusive. there is an explicit transfer of selected descriptors ( signals, timers, etc., i.e. event sources ) from a to b; there is also implicit transfer back of any "live" event sources upon exiting a, i.e. anything transferred in and still open and anything created while inside a. Consider eventScope Javascript function:

function eventScope( [ eventSource1, eventSource2, ... ], scopedFunc ) {
     var eventLoop = new EventLoop();
     eventLoop.accept( eventSource1 );
     eventLoop.accept( eventSource2 );
     // more sources transferred into new event loop

    eventLoopStack.push( currentEventLoop() );
    setDefaultEventLoop( eventLoop );

    var bDone = false;
    scopedFunc( function() { bDone = true; } ); // normal cps inside user-supplied function

    while( !bDone ) {
      runDefaultEventLoop( BLOCK_FOR_EVENTS );
    }

    var prevEventLoop = eventLoopStack.pop();
    prevEventLoop.accept( eventLoop.getNewEventSources() );
    setDefaultEventLoop( prevEventLoop );
}

so this provides a form of "selective receive", where user can directly control sequential processing order .
btw, i'm not sure if recursive/nested ticks is the right solution here; it's simply a title to grab your attention :)

thanks!

[vkurchatkin]

@eghteentwelve have a look at #7323

[eghteentwelve]

yep, so what's the plan?

[vkurchatkin]

I've made a proof of concept some time ago: vkurchatkin/node@5477d44
It works exactly as in your example apart of eventLoop.accept thing, I'm pretty sure it's impossible. Anyway, the problem is that in js-land there are too much globals, tied with main loop. To make things work the whole Environment should be created and pushed on a stack. Nothing impossible, but I don't think core team is interested at all.

[eghteentwelve]

Here is my situation: i love node.js and base my project on it; but i do feel that i'm not going to be successful unless i provide "selective receive" ( mostly due to nature of my users and type of programs they write ). I'm now looking at different ways of implementing it ( in order of my preference ):

1. node.js natively supports it; something similar to what i described above
2. i can implement add-on, but it'd most likely require many changes in node core; dealing with file descriptors and signals is pretty straightforward, it's the tick semantics that worries me
3. implement custom "standard" library to run with node; this library will not use any ticks and use cps exclusively; this will still require some changes in node C++ but not much and mostly of a good kind - making things reentrant, eliminating globals, etc.
4. using something like http://www.neilmix.com/narrativejs/doc/
5. fork node

[vkurchatkin]

@eghteentwelve what is your use case for "selective receive"?

[eghteentwelve]

nature of my users and type of programming they do ( personally, i also find it easier to reason about large programs when helped by "selective receive" ). unfortunately i cannot comment more on my use cases. thx

[eghteentwelve]

"selective receive" name comes from Erlang btw :) but the nested loop techniques are everywhere, in your modal dialog box, in your C programs that do write(); read();, in you networked file system, etc...

[vkurchatkin]

if you don't really need blocking, I think this can be done purely in javascript, maybe with some generator sugar

[eghteentwelve]

i didn't want to use the "b" word, but, yes, i need blocking call semantics. implementing it all in javascript using existing node facilities is not very high on my list, in fact it's number 4.. :-)

[jasnell]

@vkurchatkin ... any further thoughts on this?

[vkurchatkin]

@jasnell I want to return to #7323 once worker threads land. Not sure if it addresses OPs problems, but I think it is as close as we can get

[jasnell]

Ok, marking this as a defer to converged then