Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource in endless modification loop #2187

Closed
mf-lit opened this issue Mar 4, 2021 · 3 comments · Fixed by crossplane-contrib/provider-aws#576
Closed

Resource in endless modification loop #2187

mf-lit opened this issue Mar 4, 2021 · 3 comments · Fixed by crossplane-contrib/provider-aws#576
Assignees
Labels
bug Something isn't working
Projects

Comments

@mf-lit
Copy link

mf-lit commented Mar 4, 2021

What happened?

I'm observing the following issue when managing an RDSInstance with crossplane.
The resource manifest for the RDSInstance is as follows:

apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
metadata:
  name: demo-db
spec:
  providerConfigRef:
    name: crossplane-provider-aws  
  forProvider:
    engine: postgres
    engineVersion: "12.5"
    allocatedStorage: 10
    masterUsername: root
    multiAZ: false
    dbInstanceClass: db.t2.small
    region: eu-west-1
    deletionProtection: false
    publiclyAccessible: false
    storageEncrypted: false
    enablePerformanceInsights: false
    skipFinalSnapshotBeforeDeletion: true
    vpcSecurityGroupIds:
      - sg-<redacted>
    dbParameterGroupName: demo-postgres12
    dbSubnetGroupNameRef:
      name: demo-db
  deletionPolicy: Delete
  writeConnectionSecretToRef:
    namespace: crossplane-poc
    name: demo-db

Both the instance and the subnetgroup are created succesfully, but the instance constantly goes in and out of a "modifying" state.

The following event appears on the instance resource:

Normal  UpdatedExternalResource  4m41s (x78 over 40m)  managed/rdsinstance.database.aws.crossplane.io  Successfully requested update of external resource

A bit of digging, and it seems that the ParameterGroup is being "updated" on the instance every few minutes, and so the instance itself keeps flipping-in-and-out-of a Modifying state.

I don't understand why this is occurring and I'm so far unable to troubleshoot further, there's nothing logged by the provider to suggest why it's making this change. I appreciate that I should perhaps be opening this issue against the provider and not crossplane, but some general guidance about how to troubleshoot this would be appreciated, thanks.

image

What environment did it happen in?

Crossplane version: 1.0
Kubernetes version: (EKS) 1.18, kubectl 1.18

@mf-lit mf-lit added the bug Something isn't working label Mar 4, 2021
@muvaf muvaf added this to Proposed in v1.2 via automation Mar 4, 2021
@muvaf muvaf self-assigned this Mar 4, 2021
@muvaf muvaf moved this from Proposed to In Progress in v1.2 Mar 4, 2021
@muvaf
Copy link
Member

muvaf commented Mar 4, 2021

@mf-lit Thanks for opening the issue! I'll try to reproduce and get back to you.

@muvaf
Copy link
Member

muvaf commented Mar 4, 2021

some general guidance about how to troubleshoot this would be appreciated, thanks.

https://crossplane.io/docs/v1.1/reference/troubleshoot.html usually helps but in this case, I had to debug the code to see why patch here included the db parameter group name even though it's not changed.

@muvaf muvaf moved this from In Progress to PR Open in v1.2 Mar 4, 2021
@mf-lit
Copy link
Author

mf-lit commented Mar 5, 2021

Thanks for looking into this @muvaf and for the speedy PR 👏

v1.2 automation moved this from PR Open to Done Mar 5, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
No open projects
v1.2
Done
Development

Successfully merging a pull request may close this issue.

2 participants