ReplicationGroup adoption bug

[oleg-yudovich]

Describe the bug
When trying to adapt the existing replication group by applying:

apiVersion: services.k8s.aws/v1alpha1
kind: AdoptedResource
metadata:
  namespace: ***
  name: ***
spec:
  aws:
    nameOrID: ***
  kubernetes:
    group: elasticache.services.k8s.aws
    kind: ReplicationGroup
    metadata:
      name: ***

ack-elasticache-controller reconcile is failing:

"INFO adoption.adopted-reconciler starting adoption reconciliation {"target_group": "elasticache.services.k8s.aws", "target_kind": "ReplicationGroup", "namespace": "", "name": "", "generation": 1} 1.6491549222661903e+09 ERROR controller.adoptedresource Reconciler error {"reconciler group": "services.k8s.aws", "reconciler kind": "AdoptedResource", "name": "", "namespace": "", "error": "ReplicationGroup.elasticache.services.k8s.aws "***" is invalid: spec.replicationGroupDescription: Required value"}"

P.S.
I can see exactly the same error when applying replication group YAML without a description.

Expected outcome
ack-elasticache-controller should handle the adoption of the existing replication group correctly

Environment

• Kubernetes version
• Using EKS: no
• AWS service targeted: Elasticache
• Chart version: v0.0.9

[RedbackThomson]

ReplicationGroup.elasticache.services.k8s.aws "***" is invalid: spec.replicationGroupDescription: Required value

Well this is a strange behaviour. When you describe your replication group, do you see a description field on it (and it's non-nil)?

[oleg-yudovich]

ReplicationGroup.elasticache.services.k8s.aws "***" is invalid: spec.replicationGroupDescription: Required value

Well this is a strange behaviour. When you describe your replication group, do you see a description field on it (and it's non-nil)?

Hi @RedbackThomson , thanks for the quick response.
When running aws elasticache describe-replication-groups --replication-group-id ***, I can see the description in the JSON response body.

[RedbackThomson]

Hmm yeah okay I think we will need to dig into this one further, then. There may be a problem with translating the description back into a K8s object.

[oleg-yudovich]

Do you guys have the estimation on this one?
It is a blocker for us because we can't get around this issue. All the adoption logic happens on the controller side.
Many thanks for considering my request.

[RedbackThomson]

Do you guys have the estimation on this one? It is a blocker for us because we can't get around this issue. All the adoption logic happens on the controller side. Many thanks for considering my request.

I'll look into it today and gauge how big the problem is. I'll get back to you on this

[RedbackThomson]

Okay I have figured out what's happening here. We have duplicated fields between the Spec and Status of the ReplicationGroup CRD. In the Spec we have ReplicationGroupDescription and in the Status we have Description. These are both referring to the same field on the object, but one is being used only for the CreateReplicationGroup request and the other is being returned by the DescribeReplicationGroups response. Therefore, when we attempt to adopt this resource, our description from the SDK is filling Status.Description with the correct value and leaving Spec.ReplicationGroupDescription nil. When attempting to create this custom resource, the API server is failing at validation because Spec.ReplicationGroupDescription is a required field. This should be a one-line fix in the generator, and then a new version will need to be cut.

[RedbackThomson]

@oleg-yudovich https://github.com/aws-controllers-k8s/elasticache-controller/releases/tag/v0.0.18 ! Should be all sorted, now. Please try it out and let me know.