How the event loop works

[marcoliceti]

• Node.js Version: Any
• OS: Any
• Scope: documentation

The official event loop documentation is here but it has a lot of unclear parts.

The first one is this:

poll: retrieve new I/O events; node will block here when appropriate

This is unclear because:

• it uses the word "retrieve" (events), while all other phase descriptions use the word "execute" (callbacks)
• it refers to I/O, which is confusing since there is already an "I/O callbacks" phase
• what does exactly mean "node will block here when appropriate"?

So... let's move to the paragraphs where the "I/O callbacks" and "poll" phases are explained in detail.

I/O callbacks phase:

This phase executes callbacks for some system operations such as types of TCP errors. For example if a TCP socket receives ECONNREFUSED when attempting to connect, some *nix systems want to wait to report the error. This will be queued to execute in the I/O callbacks phase.

What?! Some paragraphs above it was said that:

I/O callbacks: executes almost all callbacks with the exception of close callbacks, the ones scheduled by timers, and setImmediate()

This sounds very different.

Anyways... let's move to the poll phase detailed explanation.

It starts with:

The poll phase has two main functions:

1. Executing scripts for timers whose threshold has elapsed, then
2. Processing events in the poll queue.

Again: what?!

It seems to me that point 1 is a repetition of the "timer" phase, while point 2 is a repetition of the "I/O callbacks" phase.

Note: The diagram also suggests that the poll phase relates to input only, while other parts of the article suggest that the poll phase relates both to input and to output.

Anyways, let's proceed with the "poll" phase detailed description:

When the event loop enters the poll phase and there are no timers scheduled, one of two things will happen:

• If the poll queue is not empty, the event loop will iterate through its queue of callbacks executing them synchronously until either the queue has been exhausted, or the system-dependent hard limit is reached.
• If the poll queue is empty, one of two more things will happen:
  • If scripts have been scheduled by setImmediate(), the event loop will end the poll phase and continue to the check phase to execute those scheduled scripts.
  • If scripts have not been scheduled by setImmediate(), the event loop will wait for callbacks to be added to the queue, then execute them immediately.

Once the poll queue is empty the event loop will check for timers whose time thresholds have been reached. If one or more timers are ready, the event loop will wrap back to the timers phase to execute those timers' callbacks.

Questions:

1. What happens when the event loop enters the poll phase and there are timers scheduled?
2. What happens when the system-dependent hard limit is reached?
3. It seems that callbacks for timers whose thresholds have been reached are executed after emptying the poll queue. But this was previously previously presented as function number 1 (of 2) of the poll phase, so I ask myself if this kind of callbacks is executed only at the end of the poll phase or both at the beginning and at the end.

The timer phase detailed explanation provides an interesting example:

For example, say you schedule a timeout to execute after a 100 ms threshold, then your script starts asynchronously reading a file which takes 95 ms.

When the event loop enters the poll phase, it has an empty queue (fs.readFile() has not completed), so it will wait for the number of ms remaining until the soonest timer's threshold is reached. While it is waiting 95 ms pass, fs.readFile() finishes reading the file and its callback which takes 10 ms to complete is added to the poll queue and executed. When the callback finishes, there are no more callbacks in the queue, so the event loop will see that the threshold of the soonest timer has been reached then wrap back to the timers phase to execute the timer's callback. In this example, you will see that the total delay between the timer being scheduled and its callback being executed will be 105ms.

This is what I get from this example:

• when the event loop enters the poll phase and there are timers scheduled, Node processes poll events (if any) or waits for them
• it stops doing this whenever the poll queue is empty and at least on timer's threshold is reached
• when this condition is met, Node goes back to the "timers" phase, i.e. starts a new iteration of the loop

Also, according to the detailed explanation of the "poll" phase, it seems to me that another termination condition for the poll phase is when the queue is empty and setImmediate work was scheduled. In that case Node goes on with the "check" phase.

So this is my final attempt to make sense of how the event loop actually works:

1. During the "timers" phase setTimeout and setInterval callbacks whose thresholds are reached are executed.
2. During the "I/O callbacks" phase I/O callbacks for completed (not pending) I/O are executed.
3. During the "polling" phase Node waits for pending I/O to be completed. When an I/O operation completes, it goes into the poll event queue. Node processes this events immediately and while doing this, new events may be enqueued. In other words, the queue starts empty and then grows when pending I/O completes (and becomes shorter as poll events are processed). The poll phase stops when one of these happens: A) the poll event queue is empty and one or more timers "are ready", B) the poll event queue is empty and there is setImmediate work scheduled, C) a system-dependent hard limit (mesured in number of processed events) is met. In case A the event loop goes back to the "timers" phase, while in case B or C it goes on with the "check" phase.
4. During the "check" phase any scheduled setImmediate work is executed.
5. During the "close callbacks" phase any close events are processed.

This seems to me a consistent explanation that matches most of what is said in the documentation (which instead seems inconsistent).

Is this explanation correct?

Can we try to make the documentation more clear?

[vsemozhetbyt]

cc @nodejs/documentation, @trevnorris and @techjeffharris (as per nodejs/node#4936)

[plusice]

I have one more question.
What if the queue is empty and the scheduled timers are not ready during the "polling" phase?Will event loop continue the loop and keep checking, which may occupies lots of CPU's time?

[marcoliceti]

@plusice I think that's indirectly answered in the example provided in the timer phase explanation:

For example, say you schedule a timeout to execute after a 100 ms threshold, then your script starts asynchronously reading a file which takes 95 ms.

When the event loop enters the poll phase, it has an empty queue (fs.readFile() has not completed), so it will wait for the number of ms remaining until the soonest timer's threshold is reached. While it is waiting 95 ms pass, fs.readFile() finishes reading the file and its callback which takes 10 ms to complete is added to the poll queue and executed. When the callback finishes, there are no more callbacks in the queue, so the event loop will see that the threshold of the soonest timer has been reached then wrap back to the timers phase to execute the timer's callback. In this example, you will see that the total delay between the timer being scheduled and its callback being executed will be 105ms.

In short: when the queue is empty and scheduled timers are not ready, Node.js will remain in the polling phase.

I think that the general idea is that the polling phase is about responding quickly to I/O events as they occurr. At the same time, it's important not to delay too much ready timer callbacks, so as soon as the poll queue is empty Node.js temporarily stops "worrying" about I/O and executes ready timer callbacks (if any). While these callbacks are executed, new I/O events go into the I/O callbacks queue, not the poll queue (although this distinction is probably not so relevant).

Of course this is only my current understanding. It would be nice to have some a confirmation (or a different explanation).

[plusice]

@marcoliceti Thank you for your explanation! And after reading some open source explanations of Node.js，I find that "polling phase" has a time-out.

• If scripts have been scheduled by setImmediate(), "polling" phase will set a time-out which is zero.It means that after the queue has been exhausted, "polling" phase will not wait for callbacks to be added to the queue but continue to the check phase.
• If scripts have been scheduled by setTimeout(), "polling" will set a time-out which is the result of the soonest threshold of timers minus current time.Then when time out, the loop continues and finally wraps back to the timers phase.

Actually, after reading the code I think the explanations of phase in the document make it harder to understand the process.For example, actually there are two "timers" checking in a loop.And what is called "wrap back to the timers phase" in the explanation:

Once the poll queue is empty the event loop will check for timers whose time thresholds have been reached. If one or more timers are ready, the event loop will wrap back to the timers phase to execute those timers' callbacks.

is just the second "timers" checking in the loop.We can see it from the code:

while (r != 0 && loop->stop_flag == 0) {
     // first timers
    uv__update_time(loop);
    uv__run_timers(loop);
    ran_pending = uv__run_pending(loop);
    uv__run_idle(loop);
    uv__run_prepare(loop);

    timeout = 0;
    if ((mode == UV_RUN_ONCE && !ran_pending) || mode == UV_RUN_DEFAULT) {
      timeout = uv_backend_timeout(loop);
    }

    uv__io_poll(loop, timeout);
    uv__run_check(loop);
    uv__run_closing_handles(loop);

    if (mode == UV_RUN_ONCE) {
     // second timers
      uv__update_time(loop);
      uv__run_timers(loop);
    }

    r = uv__loop_alive(loop);
    if (mode == UV_RUN_ONCE || mode == UV_RUN_NOWAIT)
      break;
  }

From the code we can find that after the callbacks executing in "polling" phase(uv__io_poll), the "check"(uv__run_check) phase will execute before "timers" phase(uv__update_time, uv__run_timers).So this code will always log "immediate" first:

var fs = require('fs');

fs.readFile(__dirname + '/' + __filename, () => {
    setTimeout(() => {
        console.log('timeout')
    }, 100)
    setImmediate(() => {
        console.log('immediate')
    })
})

So...the same appeal, can we try to make the documentation more clear?

[marcoliceti]

@plusice I found that the documentation of libuv is a great source for understanding how Node's event loop may work.

However, actual behavior depends on how libuv is used, so taking a look at Node's source code (as you did) is better.

Unfortunately, read and understand the source code is not so easy.

I think the best would be if some Node committer could help in fixing the documentation based on questions in this issue.

[plusice]

@marcoliceti can't agree more

[gireeshpunathil]

closing as answered, please let me know there is something outstanding in this.

[marcoliceti]

@gireeshpunathil Please, re-open. There are no actual answers in this thread, just hypotheses from the same people asking the questions.

[gireeshpunathil]

@marcoliceti - sure. I will see if I can explain your queries.

[Flet]

This talk helped me get a better understanding of the event loop at a high level:

Philip Roberts: What the heck is the event loop anyway? | JSConf EU 2014

It may not get to the level of detail folks are looking for here, but it helped me understand. :)

[gireeshpunathil]

@marcoliceti - I am back from vacation, so let us sort this out; if you have time. I guess we will need to run few iterations as many clarifications are required.

Do you want point-to-point answers to the questions you posted? I agree that the said doc is complex, but it makes sense, as the event loop is actually more complex than what is covered through the doc, the doc was a best effort attempt to cover as much internal detail as possible - every source line in the event loop is worth 10 lines of documentary explanation!

or do you want a basic understanding of the UV loop before even bringing the doc into the picture? there is an SOF answer I posted 2 years back when I was still learning the loop logic, but reading it now I believe that there is no incorrect information, and helps bring different perspectives: https://stackoverflow.com/questions/10680601/nodejs-event-loop/36859504#36859504

Third: One reference to the poll phase seem to have caused a lot of confusion.
https://github.com/libuv/libuv/blob/ff167ea5db66903c1a2f65b29ea0147b1108f2c9/src/unix/linux-core.c#L292

The timeout parameter that is passed to the poller primitive (uv__epoll_wait) is a key entity. As the loop has to manage both I/O workload as well as custom scheduled (deferred through timers) workload, the poller primitive is overloaded to carry both: return either with a set of I/O events that are ready to fire, or when the timeout has been elapsed - the timeout being a value carefully crafted to address any pending non-I/O deferred tasks.

If there are are no timers involved and if there are no I/O events ready at the moment, the poller primitive actually BLOCKs indefinitely, until an event has become ready.

This addresses at least 3 of your source of confusion i.e.:

poll: retrieve new I/O events; node will block here when appropriate

The poll phase has two main functions:

1. Calculating how long it should block and poll for I/O, then
2. Processing events in the poll queue.

What happens when the event loop enters the poll phase and there are timers scheduled?

I can explain more, but will wait for your response.

[marcoliceti]

Hi @gireeshpunathil . Thanks for the reply!

This addresses at least 3 of your source of confusion

Yes, it does :)

There is still a couple of things that I don't understand about the poll phase, though.

First, it's not clear if timer callbacks are executed during the poll phase or not. The doc says they are:

The poll phase has two main functions:

1. Executing scripts for timers whose threshold has elapsed, then
2. Processing events in the poll queue.

But a few paragraphs later it seems that the poll phase just detects that there are timer callbacks ready to be executed (and then the loop wraps back to the timers phase for actual execution of these callbacks). I'm talking about this paragraph:

Once the poll queue is empty the event loop will check for timers whose time thresholds have been reached. If one or more timers are ready, the event loop will wrap back to the timers phase to execute those timers' callbacks.

So, are timer callbacks executed during the poll phase or not?

Also, what is the exact role of the I/O callbacks phase?

I mean, since the poll phase both retrieves and executes I/O events, what are the conditions under which an I/O event is handled in the I/O callbacks phase instead of the poll phase?