feat(Bigtable): expose totalTimeoutMillis in RetrySettings for Read and Mutate Rows

[bshaffer]

b/391904703

Fixes the Bigtable conformance tests by ensuring that an operation timeout is set:

• TestMutateRows_Generic_DeadlineExceeded
• TestReadRow_Generic_DeadlineExceeded
• TestReadRows_Generic_DeadlineExceeded

An operation timeout (timeout for the entire operation including retries) can now be set by passing totalTimeoutMillis as part of the RetryOptions array:

// This number is passed to gRPC, and ensures a single attempt does not take longer than this number
$perAttemptTimeoutMillis = 100;

// This number is used in the library to keep track of the time spent making the attempts. If the total time
// for this operation exceeds this number, an exception is thrown
$perOperationTimeoutMillis = 3000;

$table->mutateRows($mutations, [
    'timeoutMillis' => $perAttemptTimeoutMillis,
    'retrySettings' => [
        'totalTimeoutMillis' => $perOperationTimeoutMillis,
    ]
]);

Important: I've added preempting rpc timeout in 1e152c1 which overrides timeoutMillis when it's less than the remaining time in the operation. For example, if we retry twice at 100ms and our totalTimeoutMillis is 220ms, then the third retry attempt will have a timeoutMillis of 20s. Otherwise the totalTimeoutMillis wouldn't kick in until after 300ms.

[mutianf]

should we specify in the doc that 0 means no total timeout?

[bshaffer]

We can do that, although it maybe relevant to note that in the other API methods, RetrySettings::$totalTimeoutMillis is required, and so setting it to 0 actually results in the value 0 for total timeout (which would essentially disable retries).

For that reason, it probably would make more sense for us to make -1 mean no total timeout, so that if a user passes in 0 in both cases, the behavior is consistent. What do you think? Do you know what other language libraries are doing?

[mutianf]

I know in java 0 means no total timeout

[bshaffer]

Then unless it's a requirement for this feature (which maybe it is?) I'd rather leave it consistent with the rest of our RetrySettings

[mutianf]

Ok I think we can keep the current behavior and just add a comment to call it out

[bshaffer]

note: this means the default is no retries, which we probably don't want to do

[mutianf]

what's the current behavior?

[bshaffer]

@mutianf Ping on this, I'd love your LGTM if possible!