Generate deadline per call from single configuration

[richardstartin]

What version of gRPC are you using?

1.9.0

What did you expect to see?

The ability to specify a desired timeout per stub and have a deadline generated from the specified timeout per RPC.

Unless I'm missing something, the only way to effectively specify a (relative) timeout is to use (absolute) deadlines repetitively as follows:

MyServiceStub stub = newBlockingStub(channel);
stub.withDeadline(Deadline.after(100, TimeUnit.MICROSECONDS)).foo(request);

This means I need to write .withDeadlineAfter(100, TimeUnit.MICROSECONDS) or similar on every call site. It would be nice to be able to specify a timeout or deadline generation specification on the stub since I consider it a cross cutting concern.

MyServiceStub stub = newBlockingStub(channel).withDeadlineSpecification(100, TimeUnit.MICROSECONDS);
...
stub.foo(request);

Where a Deadline equal to Deadline.after(100, TimeUnit.MICROSECONDS) has been set on the CallOptions before each call.

Is it otherwise possible to specify a relative cutoff?

[carl-mastrangelo]

This is expected. Stubs are meant to be forked, and it is fairly cheap to do so. It is expected behavior.

You can accomplish a new deadline for each call by using a ClientInterceptor:

    ClientInterceptor ci = new ClientInterceptor(){
      @Override
      public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(
          MethodDescriptor<ReqT, RespT> method, CallOptions callOptions, Channel next) {
        return next.newCall(method, callOptions.withDeadlineAfter(100, TimeUnit.MICROSECONDS));
      }
    };

    AbstractStub<?> timeoutStub = myStub.withInterceptors(ci);

[richardstartin]

Sure, I can fork them, but I don’t want to because I have, say, 50 clients and 5 methods per client. It’s a question of useability.

I can go ahead and implement an interceptor as suggested, but it seems like a lower level API than I would expect. I can do real damage inside an interceptor callback, all I wanted was to be able to say “please abort all RPCs after 100us”

Thanks for the code snippet by the way.

[carl-mastrangelo]

The use case for deadlines is slightly different than the one you want, which is why the API seems wonky. Consider a chain of servers which each need to send an RPC to the next one. A person sends an RPC to Server A, which in turn sends an RPC to Server B, which finally send an RPC to server C. The call chain then returns back up to the user who gets the response.

In this case, Server A wants to say "I am interested in the answer for the next 5 seconds, after which give up". This is propagated to Server B (in the call headers), which sees "I am interested in the answer for the next 4.9 seconds", which then is propagated to C which sees 4.8 seconds. There is an absolute point in time where further RPC fan out would be wasted, so all the servers end their calls all at once. Hence, gRPC favors absolute deadlines in code.

It is also easier to reason about with absolute point-in-time deadlines, because relative delays aren't always clear. For example, if you need to get an auth token to attach to an RPC, does getting the token count towards the overall RPC timeout? Or, if there was a 10 second GC pause between your application starting an RPC and the data actually getting to the socket, does that count? I think reasonable people will have different answers to such questions. The deadline approach solves these all. (at the cost of forking the stub each time).

As for useability: can you make one root stub with the interceptor and fork it for each client? You have to do this anyway when making a stub out of a channel.

[richardstartin]

Indeed, my use case is simple but ubiquitous: guarding against cascading failure by limiting all RPCs to a fixed duration. I do understand why deadlines are used internally, and do appreciate the length you have gone to in order to convey this, but I still consider it an implementation detail passed on to the caller.

Given a timeout specification and a clock, I can generate as many deadlines as I like, but it's not a concise representation of information. This API forces users to repeat themselves by forking the stub per RPC invocation, rather than accepting a timeout specification once and then generating the deadline on the caller's behalf at the point of invocation. In fact, your arguments in your third paragraph could only support this (that is, generating the deadline at the last possible moment).

Finally, does this issue come up a lot?

[carl-mastrangelo]

@richardstartin In the predecessor of gRPC, all time specifications were relative (which I will call 'timeouts' as opposed to 'deadlines'). Additionally, they were specified in the proto file in the method definition. This absolved people from having to pick a timeout ever, except if they really wanted to. Theoretically, you can do this today with an interceptor (using proto reflection on the MethodDescriptor).

Given a timeout specification and a clock, I can generate as many deadlines as I like, but it's not a concise representation of information.

Realistically, only the root in the RPC fanout hierarchy needs to do this. The rest of dependent RPCs can only pick sooner deadlines. This is rare. I don'y know how many RPC callsites you have, or how often you make stubs, but on the surface it seems like you only need to do this once in your code:

MyServiceStub stub = 
   newBlockingStub(channel).withInterceptor(new TimeoutInterceptor(100, TimeUnit.MICROSECONDS));

In fact, your arguments in your third paragraph could only support this (that is, generating the deadline at the last possible moment).

That's true, but it would be ambiguous otherwise. We aren't in a position to make a good guess about what the user wanted. There is a long list of cases where a relative deadline would be confusing or astonishing. I gave the previous two examples as cases where we cannot guess. Here are some more:

• If the channel is in flow control pushback, and the RPC has not yet been sent, does the timeout start when it actually goes on the wire or when ClientCall.start is called?
• If the timer starts when the first byte of the RPC hits the wire, when is that exactly? The socket API doesn't tell you if the data ever arrived. It could have been passed to the kernel, but sat in a buffer and never went out.
• When is the timeout over? Is it upon getting the first byte back from the remote endpoint? Is it after getting back initial headers? After the response payload? After trailers? What if there are long delays between these events?
• In the case of streaming, what does it mean to have a timeout? There could be many payloads being sent. Is it the timeout between payloads? And does it get reset each payload?

With a deadline, there is a clear and understandable answer. It may not be the one you want, but you will know for sure what's going to happen. You don't need to read the gRPC source code to find out.

Finally, does this issue come up a lot?

Not really. Most people just don't want an infinite timeout.