Allow configuring custom timeouts for Create, Update, Delete operations on a per-resource basis

[lukehoban]

There are scenarios where the default timeouts in the provider are insufficient. In order to allow users to not be blocked, we should allow configuring custom timeouts.

[naveensrinivasan]

@lukehoban When is this likely to be implemented?

[lukehoban]

@naveensrinivasan We're hoping to be able to tackle this as part of the current milestone.

[stack72]

So I have been looking at how we will do this over the past few days. The user part is going to look as follows (using AWS DynamoDB Table as an example because Terraform supports customTimeouts):

import * as aws from "@pulumi/aws";

const table = new aws.dynamodb.Table(
    "testtable",
    {
        attributes: [{
            name: "id",
            type: "S",
        }],
        hashKey: "id",
        readCapacity: 5,
        writeCapacity: 5,
        streamEnabled: true,
        streamViewType: "NEW_AND_OLD_IMAGES",
    },
    {
        customTimeouts: {
          create: "10m",
          delete: "10m",
          update: "30m",
        },
    }
);

This means we will be extending the ResourceOpt struct to support this new option that we can pass to the resource.

@ellismg / @lukehoban / @jen20 I'd love for you guys to have your thoughts on this - as this will be forming the basis of work here.

[lukehoban]

@stack72 That looks right to me.

I assume this will get passed directly through to the providers, and they will have to figure out how to implement them? (That is, that this is not a feature of the engine like e.g. ignoreChanges was, it's a feature of providers). And that we will need to figure out how to implement using the similar Terraform capability for Terraform-based providers, and how to implement natively against Kubernetes and other raw Pulumi providers.

Notably, I don't believe we've had any resource options like this so far that are passed through to providers. So far, all the resource options have been interpreted by the Pulumi engine. This raises some questions around what should happen if the provider can't implement the requested custom timeouts, whether the full set of configuration here is going to look sufficiently the same across all providers, etc. I think the answer is that we do want a consistent model to present across all providers, and that extending the provider RPC interface to pass these custom timeouts down is the right thing to do.

cc also @pgavlin for thoughts

[pgavlin]

I assume this will get passed directly through to the providers, and they will have to figure out how to implement them

I think that's going to be necessary, yes. TF will certainly need to take this approach.

Notably, I don't believe we've had any resource options like this so far that are passed through to providers. So far, all the resource options have been interpreted by the Pulumi engine. This raises some questions around what should happen if the provider can't implement the requested custom timeouts, whether the full set of configuration here is going to look sufficiently the same across all providers, etc. I think the answer is that we do want a consistent model to present across all providers, and that extending the provider RPC interface to pass these custom timeouts down is the right thing to do.

I agree. Even if certain providers are unable to implement the custom timeouts, I do think that plumbing this through the RPC interface is the right move. Perhaps we can generate static typings or early errors that indicate that custom timeouts are unavailable for certain resources.

[lblackstone]

Related to pulumi/pulumi-kubernetes#457

I added timeout logic for the k8s provider using annotations, but this would be useful as a more general-purpose way of setting that.

[clstokes]

Here are some links for anyone that comes across this in the future:

• Custom Timeouts are now documented at https://www.pulumi.com/docs/intro/concepts/resources/options/customtimeouts/
• An example that uses it - https://github.com/pulumi/examples/blob/0ae4ddb3663f90aeb4fac9642f6385fd0356f6ec/aws-ts-pulumi-miniflux/index.ts#L66-L69.

[phillipedwards]

For anyone looking for where Custom Timeouts have been moved to: https://www.pulumi.com/docs/intro/concepts/resources/#options

[reubano]

I've been trying Custom Timeouts with azure-native and have found that (compared to not using CustomTimeouts) it swallows the error messages and puts the stack in an inconsistent state ("cannot create already existing resource"/"is still transitioning").