Configurable timeout for resource type checks

[jamieklassen]

As #2352, but for resource type checks.

Two problems when it comes to testing/reasoning about this feature -- how do you know that a resource type check happened, and how do you know how it went?

did it happen?

One of the things we learned while implementing #2352 was that using fly check-resource was a helpful way to test in testflight. However, at present there is no fly check-resource-type (or corresponding API endpoint) and so testing could be difficult/expensive. On Discord this feature was discussed/requested among @billimek @vito and @Typositoire -- it may be a reasonable prerequisite for this story.

how did it go?

Along the same theme of "difficulty of observing resource type checks" at present we're concerned about how to assert on the success/failure and side effects of a resource type check. Again in #2352 we were able to use fly hijack -c to dive into the container that performed the check to assert about side effects, but I'm not sure that you have the same option for resource type checks.

[Typositoire]

Well I asked this at first because it's really hard right now, unless you're tailing ATC logs to know WHICH resource type version is actually being used and when it's updated (most of resource type use :latest). So when debugging having a command to manually force a new update of a given resource type would have been nice. But I'd be really happy with just a panel/dashboard or command that can track the states of a given resource type or all of them.

I dunno if this would be much less work than having the full feature of checking the resource type but it'd be happy with only this for a while.

The configurable check timeout is just a work around the fact that we don't know (easily) what's happening for a given resource type. By lowering it I would have able to be blind for a shorter amount of time :p

[pivotal-saman-alvi]

In order to complete or test this story we will need to be able to be able to trigger resource type checks. Currently there is a fly check-resource flag and corresponding ATC handlers. In order to proceed it would be helpful to first implement the Fly/ATC API for check-resource-type. That is split off into: #2507

[vito]

@clarafu and I are actually discussing as part of #2386 whether we want to just merge resources and resource_types, and allow resources to refer to other resources as their "type" instead of implementing entirely symmetrical feature sets for both. We haven't made a decision yet as to whether we want to try to do this backwards-compatibly or wait for 5.0.

[vito]

Let's use the same value from resource check timeouts for types as well.

[clarafu]

Closing this because of #3553