feat(runtime): bump runtime to latest version for paused annotation

[haarchri]

Signed-off-by: haarchri chhaar30@googlemail.com

Description of your changes

added latest crossplane-runtime version to add paused annotation for resources

apiVersion: efs.aws.crossplane.io/v1alpha1
kind: FileSystem
metadata:
  annotations:
    crossplane.io/external-create-pending: "2022-10-02T10:58:03Z"
    crossplane.io/external-create-succeeded: "2022-10-02T10:58:04Z"
    crossplane.io/external-name: fs-0e6772a4e20d76a8c
    crossplane.io/paused: "true"
  creationTimestamp: "2022-10-02T10:58:03Z"
  finalizers:
  - finalizer.managedresource.crossplane.io
  generation: 1
  name: example
  resourceVersion: "515627"
  uid: e289916c-6842-417e-972d-5e49d9a54059
spec:
  deletionPolicy: Delete
  forProvider:
    region: us-east-1
  providerConfigRef:
    name: default
status:
  atProvider:
    creationTime: "2022-10-02T10:58:04Z"
    creationToken: e289916c-6842-417e-972d-5e49d9a54059
    fileSystemARN: arn:aws:elasticfilesystem:us-east-1:xxx:file-system/fs-0e6772a4e20d76a8c
    fileSystemID: fs-0e6772a4e20d76a8c
    lifeCycleState: available
    numberOfMountTargets: 0
    ownerID: "xxx"
    sizeInBytes:
      value: 6144
      valueInIA: 0
      valueInStandard: 6144
  conditions:
  - lastTransitionTime: "2022-10-02T11:18:39Z"
    reason: ReconcilePaused
    status: "False"
    type: Synced
  - lastTransitionTime: "2022-10-02T10:59:06Z"
    reason: Available
    status: "True"
    type: Ready

Fixes #

I have:

• Read and followed Crossplane's contribution process.
• Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

[haarchri]

@ulucinar is it normal that the paused annotation will cause a CannotObserveExternalResource Events? and so many ReconciliationPaused Events ?

  Normal   ReconciliationPaused             24s (x404 over 5m26s)  managed/filesystem.efs.aws.crossplane.io  Reconciliation is paused via the pause annotation
  Warning  CannotObserveExternalResource    7s (x16 over 5m7s)     managed/filesystem.efs.aws.crossplane.io  failed to describe FileSystem: ThrottlingException: 
           status code: 400, request id:

[ulucinar]

@ulucinar is it normal that the paused annotation will cause a CannotObserveExternalResource Events? and so many ReconciliationPaused Events ?

  Normal   ReconciliationPaused             24s (x404 over 5m26s)  managed/filesystem.efs.aws.crossplane.io  Reconciliation is paused via the pause annotation
  Warning  CannotObserveExternalResource    7s (x16 over 5m7s)     managed/filesystem.efs.aws.crossplane.io  failed to describe FileSystem: ThrottlingException: 
           status code: 400, request id:

Hi @haarchri,
These do not look normal to me. Especially the repeating CannotObserveExternalResource events as we should not be passing control to the provider's managed.ExternalClient implementation. I will try to reproduce it using the manifest provided. Is the provided manifest enough to reproduce this behavior?

[haarchri]

Yes it's enough - thanks for taking care

[ulucinar]

Hi @haarchri,
I did some observations with the branch of this PR by locally running the provider built from it. It looks like it works as expected. You can find the observations below.

I suspect that there is another client updating the managed resource though. Then we would observe the repeating ReconciliationPaused events. But then I would assume it's a non-backed-off fast client so that it would be updating the managed resource 400 times in 5m. This could be any API server client (a shell script, another controller, etc.) What do you think? Are there any clients, controllers acting on the managed resource? How does kubectl get filesystems.efs -o yaml -w behave?

Only the provider running locally (built from this PR's HEAD), directly provisioning the resource with the pause annotation works as expected (the only event is the ReconciliationPaused event) and reconcile requests are not re-queued:

Events:
  Type    Reason                Age                  From                                      Message
  ----    ------                ----                 ----                                      -------
  Normal  ReconciliationPaused  5m5s (x2 over 5m5s)  managed/filesystem.efs.aws.crossplane.io  Reconciliation is paused via the pause annotation

Then, when I removed the annotation, the resource has been provisioned successfully with the following events:

Events:
  Type    Reason                   Age                    From                                      Message
  ----    ------                   ----                   ----                                      -------
  Normal  ReconciliationPaused     8m21s (x2 over 8m21s)  managed/filesystem.efs.aws.crossplane.io  Reconciliation is paused via the pause annotation
  Normal  CreatedExternalResource  83s                    managed/filesystem.efs.aws.crossplane.io  Successfully requested creation of external resource

If I prevent observation from succeeding without the pause annotation:

Events:
  Type     Reason                         Age                  From                                      Message
  ----     ------                         ----                 ----                                      -------
  Warning  CannotObserveExternalResource  38s (x4 over 3m38s)  managed/filesystem.efs.aws.crossplane.io  failed to describe FileSystem: RequestCanceled: request context canceled
caused by: context deadline exceeded

If I prevent observation from succeeding with the pause annotation:

Events:
  Type     Reason                         Age                 From                                      Message
  ----     ------                         ----                ----                                      -------
  Warning  CannotObserveExternalResource  8m4s (x5 over 12m)  managed/filesystem.efs.aws.crossplane.io  failed to describe FileSystem: RequestCanceled: request context canceled
caused by: context deadline exceeded
  Normal  ReconciliationPaused  7m48s (x3 over 8m4s)  managed/filesystem.efs.aws.crossplane.io  Reconciliation is paused via the pause annotation

[haarchri]

@ulucinar nice Catch - you are correct i have running a other provider in the cluster - then LGTM @muvaf @ulucinar please approve ;)

[ulucinar]

Thank you @haarchri, lgtm.