Feature request for controller to acknowledge service networks and services managed by other accounts

[shulin-sq]

Is there a recommended configuration for the k8s controller when lattice services are expected to be shared across accounts?

In an example scenario:
I have account A and account B

• account A uses kubernetes, account B uses ec2 instances. Each account has its own lattice service network already configured.
• account A uses the k8s controller to manage lattice services, account B uses terraform/manual configuration
• account A wants account B to be able to call its lattice service A
• account A shares lattice service A with account B
• account B associates service A with service network B -- but this here it seems the k8s controller does not like this.

  aws-application-networking-k8s/pkg/deploy/lattice/service_manager.go

  Lines 271 to 283 in 3927a6e

  	// go through SVC's association list
  	// check SN is in desired SN list
  	// if NOT, delete svc->SN association
  	// TODO logic here
  	listServiceNetworkServiceAssociationsInput := vpclattice.ListServiceNetworkServiceAssociationsInput{
  	ServiceIdentifier: &svcID,
  	}
  	resp, err := latticeSess.ListServiceNetworkServiceAssociationsAsList(ctx, &listServiceNetworkServiceAssociationsInput)
  	glog.V(6).Infof("ListServiceNetworkServiceAssociationsAsList req %v, resp %v err %v",
  	listServiceNetworkServiceAssociationsInput, resp, err)
  	for _, snAssocResp := range resp {

  per this code, the controller will delete the service<>service network association since it's not captured in the HTTPRoute of service A

I know there is the annotation that makes kubernetes ignore vpc associations to a gateway[0]. However even with this, the gateway/service network is still created in account A. In this scenario, account B wants to manage its own infrastructure as much as possible. With the current limitations it seems that all lattice resources must be created in 1 account and then shared out to other accounts.

Some possible solutions I can think of

1. consider foreign resources a special case such as having an option to force the controller to not delete a service<>service network association if the account ids of these resources are different
2. similar to the previous idea, allow a reference to a non-existent gateway which refers to a remote service network (on another account)
3. allow k8s controller to manage resources for multiple accounts. (I like this option less since it gives less independence to other accounts to manage their own AWS resources)

I am definitely open to other ideas I am not considering and would be happy to add more context
[0] https://github.com/aws/aws-application-networking-k8s/blob/main/docs/multi-sn.md

Note: after discussion with @liwenwu-amazon on slack is sounds like prefer solution 1

In her words: "lattice controller not to disassociate lattice service from a lattice service network , if that lattice service network owned by a different account"

[shulin-sq]

To add to this: we also want to support the other direction, where account B shares a service to be associated with account A's service network.

[thetkoseek]

Adding to this. If Account A is running eks on a shared vpc, then the annotation application-networking.k8s.aws/lattice-vpc-association will not work correctly as you cannot associate a shared vpc with a service network. It throws the following error AccessDeniedException: Vpc ownership validation failed.

The solution to this is to allow services shared to account b to be associated with the service network in account b.

[zijun726911]

One possibly way to implement solution1:

• Grab serviceNetwork owner accountId by ServiceNetworkServiceAssociation’s serviceNetworkArn from ListServiceNetworkServiceAssociations response at line 278,

• At line 288, do an extra condition check, if serviceNetworkOwnerAccount != currentControllerUsingAccount (or just serviceNetworkOwnerAccount != serviceOwnerAccount? ), serviceNetworkAssociationMgr just ignores to delete that ServiceNetworkServiceAssociation