Skip to content

ServiceExport's Pod IPs mutation don't work, serviceexport_controller misses to watch Endpoints #477

@zijun726911

Description

@zijun726911

If the user use k8s Service to refer some pods and use ServiceExport to create lattice Target Group for it, if Pod IPs addresses change, The controller can NOT register new and deregister stale ip addresses in the VPC Lattice target group

How to reproduce it:

  1. kubectl apply -f examples/inventory-ver2.yaml (which includes a Deployment and a Service)
  2. kubectl apply -f examples/inventory-ver2-export.yaml (which includes a ServiceExport)
  3. VPC Lattice target group and its targets created as expected
  4. do kubectl get endpoints inventory-ver2, all endpoints IP addresses === target group's targets IP addresses
  5. do kubectl rollout restart deployment inventory-ver2 to re-create all pods of inventory-ver2
  6. do kubectl get endpoints inventory-ver2, all endpoints IP addresses change to new ones
  7. However, this IPs change did not reflect to target group's targets IPs change, that is controller bug

Pod IPs mutation for k8s Service could work (i.e., a route's targetref's Service created target group), because the controller Watches Endpoints here

However, don't work for ServiceExport ( i.e, a ServiceExport created target group) probably because the controller MISSES to Watches Endpoints at serviceexport_controller.go

That could be the root cause of this customer facing issue:
#476

Restarted the deployment 1. Looks like it is taking forever for the controller to detect the new IPs and update the target group with those IPs, as a result targets stay unhealthy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions