Adding an interface to query readiness to accept requests

[lann]

This would serve a few purposes:

• As an instance reuse "handshake" between the caller and handler, allowing e.g. a host to drop an instance that won't accept any further requests or a guest to eagerly perform some expensive state reset.

• As a form of flow control (back pressure) for entire requests. This is useful in load-balancer-like situations where the sender wants to pick from only handler(s) that are ready to process a request.

• A way for handlers to explicitly signal concurrency support.

A rough draft:

interface readiness {
  /// May be called to query the handler's readiness to accept request(s).
  /// - Blocks if the handler is not ready to accept any more requests but may become ready in the future.
  /// - Returns `ok(n)` when the handler is ready to accept up to `n` concurrent requests.
  /// - Returns `err` if this instance won't ever accept another request (instead of exiting).
  /// - Returns `ok(0)` if the handler cannot report its readiness; the caller will just have to take
  ///   its chances on `handle` blocking (maybe better as an error variant?).
  ready: async func() -> result<u32>;
}

This would be added to the proxy world or the proposed service world. Alternatively this function could be added to the handler interface, though I'm not sure that it is applicable to generic "outbound http" imports.

[tschneidereit]

I've thought about this a bunch and talked about it with @dicej and others. So far, my conclusion has been that we already have what we need:

• allowing e.g. a host to drop an instance that won't accept any further requests

I think the right way for a guest to signal that is to exit(0).

• A way for handlers to explicitly signal concurrency support.

Is your thinking that this would make it so a guest is forced to make a statement about concurrency support, compared to backpressure signaling? Because apart from that it seems like backpressure signaling does what we need here, no?

I think the bar for introducing this should be pretty high at this point, because ideally we wouldn't introduce new things to p3 interfaces anymore.

[lann]

I've thought about this a bunch and talked about it with @dicej and others.

This is stemming from a conversation with @dicej and others. 🙂

I think the right way for a guest to signal that is to exit(0).

I think there was a concern that this is not composable, i.e. just because some nested component is done doesn't mean the entire composition must be done.

Is your thinking that this would make it so a guest is forced to make a statement about concurrency support, compared to backpressure signaling? Because apart from that it seems like backpressure signaling does what we need here, no?

My understanding is that backpressure is only visible once an export is actually called, which would force the caller to "commit" to making the request (or copy/buffer it). I'm also unsure if the caller can distinguish between a call blocking due to backpressure vs any other reason.

[tschneidereit]

This is stemming from a conversation with @dicej and others. 🙂

I was already sad I couldn't make it to that meeting, and now I'm even more so :/

I think there was a concern that this is not composable, i.e. just because some nested component is done doesn't mean the entire composition must be done.

Hmm. To the degree that really is a problem, how would this proposal address it? It seems like it'd still require the top-level component (or, if that's not a thing as such, whatever component exports the proposed function) to make the decision. It might consult other composed components in doing so, but it could do that before calling exit(0), too.

Obviously the fact that any nested component can call exit(0) at any time is a composability issue, but that seems to be neither here nor there.

My understanding is that backpressure is only visible once an export is actually called, which would force the caller to "commit" to making the request (or copy/buffer it). I'm also unsure if the caller can distinguish between a call blocking due to backpressure vs any other reason.

That used to be the case, until the introduction of backpressure.{inc,dec}.

[lann]

Another approach which would capture some of this functionality would be to wrap the handle function in a resource (similar to this "endpoints" idea) and use the async function returning that resource to signal readiness, e.g. get-endpoint: async func(name: string) -> result<endpoint>, or even a stream<endpoint>.

My main concern is that if this mechanism is anywhere other than the proxy world (or whatever world ends up best representing "middleware") then its usefulness will be capped by whatever portion of middleware implementations choose to opt-in to it.