Make Ask less expensive

[Horusiath]

Current state

Right now the way, how Ask is realized internally upon call, is that we're creating a temporary lightweight actor instance instance, that is supposed to asynchronously await for the response on our behalf.

This means, that we need to:

1. Allocate actor - even if it's lightweight, it's still not alloc-free operation.
2. Register/unregister temporary actor every time, so that it can be accessed by it's path (necessary to make Ask working with remote transparency mechanics). This also takes time.

All of that makes our Ask calls orders of magnitude slower and more expensive than standard Tell communication.

Idea

We could possibly change the mechanics behind Ask method to make use of pooling together with ValueTask<T>: it's an awaitable value type (convertible to Task<T> via AsTask method).

ValueTask<T> allows to use special constructor in form:

ValueTask<T>(IValueTaskSource<T> notifier, short correlationId)

This way it can be used together with pooling mechanisms, to make async await work without allocating TaskCompletionSource every time.

Example usage is new .NET socket I/O - SocketAsyncEventArgs are implementing this interface, making methods like Socket.SendAsync possible to work without any extra allocations.

How we could use this in akka? The idea is: instead of registering/deregistering Ask listener actors on every Ask call, we could keep a (dynamically resizable) pool of registered listeners, that could be reused across many Ask calls.

In order to correctly deadletter overdue responses - which may have arrived after ask has been timed out or cancelled, and actor was reused for another request - we have a second ValueTask parameter, which is correlationId.

This would mean, that we'd need to keep correlationId in Ask listener and pass it through back and forth. It's pretty easy in local communication (we already have such "magic number" in the Envelope itself), but probably would require something extra in terms of remote communication (maybe we could reuse ActorPath UIDs for that?).

[Aaronontheweb]

I think it'd be possible to have a ThreadLocal pool of re-usable Ask actors that can be "reset" back to an empty set and re-used. I don't know about the IValueTaskSource<T> because I haven't worked with it at all hardly. Open to the idea.

[Bishnup1995]

Hey Horusiath,
Did you find any way to avoid creating of internal actor for every ask request?