API Overhaul for CAF 1.0: Special Handlers

[Neverlord]

Starting this as a discussion here to have big-picture discussion instead of multiple discussions on details in tickets.

Not all messages are created equal. CAF has a few message types that bypass the regular actor behavior. In some cases (like streaming), this is fully transparent to the user. In other cases, users can still override the default processing. For this, CAF currently has a couple of special handlers. Namely:

• default_handler for catching messages that aren't matched by the behavior
• error_handler for processing error messages
• down_handler for processing down_msg messages
• node_down_handler for processing node_down_msg messages
• exit_handler for processing exit_msg messages

I think it's time to revisit some of the design decisions that lead to this implementation. Our toolkit in CAF is much richer than it has been in the past, especially due to action and disposable.

As pointed out by @dominiklohmann (#1423), the monitor API is quite clumsy at the moment. Mostly for historic reasons, really. The current API based on monitor and demonitor with down_msg signaling goes back all the way to the beginning alongside the design for exit_msg. However, down messages and exit messages are very different in nature. Down messages are explicitly requested by an user and users could just tell CAF what to do in case of the other actor going down at the point of calling monitor. Exit messages on the other hand can happen if another actors decides to link against an actor. Hence, basing monitor on the new actions seems much more natural. Instead of having a separate demonitor, users can simply call dispose on the action and forget about it. With the current implementation, there's an inherent race when calling demonitor that sometimes will lead to receiving a down_msg that is no longer relevant. So to cut a long story short, I think we can deprecate down_handler and simply use the suggested API from the ticket. I'd also consider the same API for node down messages.

Regarding the default handler: we recently added a feature to CAF that allows a behavior to simply have a callback taking a message argument as a catch-all rule. The one "magic trick" that the default handler can pull off is stashing. I think this should receive it's own API instead. See #1474. Basically, instead of installing a skip handler, explicitly add a catch-all case and put messages to a caf::stash (does not exist yet) that will allow flushing messages back to the mailbox whenever the user wants to. Doing it this way, it's much easier to understand what's going on. Users will also have a way of checking how full the stash actually is to make sure it's not exploding.

The error_handler gets called for error messages. The most common way actors will receive errors is as a response to some sent message or when sending a message to an actor that drops it. The default is call quit quit(reason). Errors probably should stay separate. It's basically an "exception" case that shouldn't go through the regular behavior.

Last but not least: exit_handler. My initial thought was to deprecate this as well and use the regular behavior. However, that would make the semantics of things like request(...).await(...) a bit fuzzy. Using await suspends the regular actor behavior. However, we always need to respond to system messages such as exit_msg. So either we make an exception for exit_msg and call the suspended behavior regardless or we have a special handler for it (status quo). The latter probably makes more sense, because we don't want to have a "default-case" handler from the regular behavior processing an exit_msg. That would most likely result in tons of bugs resulting from discarding exit messages from default handlers. Same reasoning applies to the error_handler. So it's either keeping the special handlers for exit and error messages (my preference at the moment) or have them go to the regular behavior but exclude them from the default-case handler and call the behavior for these two types even if the behavior is suspended... sounds a bit convoluted to me.

TLDR, I'm thinking of these changes to the API with CAF 1.0:

• default_handler => deprecate; users should use [](const message&) { ... } in their behavior to catch any regular message not covered by other handlers and use an explicit caf::stash class if they need to skip over certain messages to process them later on
• down_handler => deprecate; replace with monitor(handle, callback) interface
• node_down_handler => deprecate; replace with monitor(node, callback) interface
• error_handler leave as-is
• exit_handler leave as-is

As always, feedback most welcome. 🙂

[dominiklohmann]

The only one here I disagree with is the error handler. I think errors should just be ordinary messages, and not system messages, especially in the context of #1584. However, I base this only off of my experience working in a codebase where all actors are required to be typed actors, and all requests must have an explicit error handler.

I think the following should apply (including the exception handler, which isn't included in the initial post):

// The exit handler is a system message handler.
self->set_exit_handler([self](const caf::exit_msg& msg) {
  // exit handler
  self->quit(msg.reason);
});

// The exception handler is a system message handler.
self->set_exception_handler([self](std::exception_ptr err) {
  try {
    std::rethrow_exception(err);
  } catch (...) {
    // exception handler
  }
});

// Down handlers are system message handlers.
self->monitor(hdl, [self](const caf::error& err) {
  // down handler
});

// All message handlers defined in the behavior are ordinary message handlers.
auto my_actor(event_based_actor* self) -> behavior {
  return {
    [self](...) {
      // user-defined handlers
    },
    [self](const caf::error& err) {
      // error handler
    },
    [self](const caf::message& msg) {
      // catch-all handler
    },
  };
}

// Request response handlers are also just ordinary message handlers.
self->request(...).then(
  [self](...) {
    // result handler
  },
  [self](const caf::error& err) {
    // error handler
  });

There is actually one further special handler that currently is in a really bad place, which is caf::after. See #1467 for a previous ticket I opened on it. I think it should also be a system message, and work something like this:

// Idle handlers are system message handlers.
self->set_idle_handler(timeout, [self] {
  // idle handler
});

[Neverlord]

The only one here I disagree with is the error handler. I think errors should just be ordinary messages, and not system messages, especially in the context of #1584.

How error messages are treated actually depends. Error messages in a request/response communication are delegated to the response error handler or fall back to calling quit(reason) if none is defined.

The reason we don't treat error messages as ordinary messages is that they represent an application error that shouldn't be ignored. Akin to an exception/panic.

When sending an unexpected message to another actor, it'll respond with an error. If this was a request, then the corresponding error handler should take care of it. If not, the sender should terminate by default. If we would dispatch the error message to the default handler, it would essentially silently discard errors. And the default would be to send the other actor an error back again. I think having actors terminate by default on unexpected errors is a good default.

Can you elaborate on the link to #1584? Is the concern that an error could basically "cancel" the response handler when it arrives before the response?

... including the exception handler, which isn't included in the initial post ...

Good point, this also exists. I didn't mention this handler, because it's not about messages. This handler allows customizing how an actor deals with uncaught exceptions from any of its message handlers.

There is actually one further special handler ... which is caf::after.

Separating the timeout from the behavior is an interesting idea. How do you think this should work? Should it be automatically cancelled when overriding the behavior via self->become(...) or would you see this as completely orthogonal?