[aws-eks] private endpoint doesn't work with looked up vpc

[iliapolo]

It's impossible to configure a private access endpoint when using a looked up vpc (i.e Vpc.fromLookup).

Reproduction Steps

1. Create a VPC using the following guide: https://docs.aws.amazon.com/batch/latest/userguide/create-public-private-vpc.html

2. Deploy the following:

const vpc = ec2.Vpc.fromLookup(this, 'Vpc', {
  vpcId: '<VPC_ID>'
})

const cluster = new eks.Cluster(this, 'Cluster', {
  vpc: vpc,
  version: eks.KubernetesVersion.V1_16,
  defaultCapacity: 2,
  endpointAccess: eks.EndpointAccess.PRIVATE
});

cluster.addResource('config-map', {
  kind: 'ConfigMap',
  apiVersion: 'v1',
  data: {
    hello: 'world',
  },
  metadata: {
    name: 'config-map',
  },
});

What did you expect to happen?

Expected for the cluster to created and for the config map to be correctly applied

What actually happened?

Environment

• CLI Version :1.57.0
• Framework Version:1.57.0
• Node.js Version:
• OS :
• Language (Version):

Other

Following is an excerpt from CloudFormation resource for the kubectl handler, we can see that the subnet id's are indeed empty:

    "Handler886CB40B": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
        ....
        "VpcConfig": {
          "SecurityGroupIds": [
            {
              "Ref": "referencetoekslookedupvpcPlaygroundClusterKubectlProviderSecurityGroupBFF8A75BGroupId"
            }
          ],
          "SubnetIds": []
        }
        ...
      },
    },

---

This is 🐛 Bug Report

[iliapolo]

After some investigation, it seems that the problem is:

aws-cdk/packages/@aws-cdk/aws-eks/lib/cluster.ts

Line 1052 in 9ff55ae

privateSubnets.push(...this.vpc.selectSubnets(placement).subnets.filter(s => s instanceof ec2.PrivateSubnet));

For looked up VPC's the type of the class is not PrivateSubnet, which results in the filtering returning an empty list.

[JasperW01]

Hi @iliapolo , thx a lot for raising this. really appreciate.

Just my 2 cents on the subnet selections for the kubectl handler Lambda given the enterprise context:

i. In most cases, we will configure vpcSubnets when calling eks.Cluster or FargateCluster to create eks clusters, so we can make sure the ENIs of the EKS control plane will be put into the subnets as required. But regardless the vpcSubnets is set or not, I guess by default the kubectl handler Lambda can just be associated with the subnets associated with the EKS Control plane.

ii. However, that might not be enough. This kubectl handler Lambda fundamentally is applying kubectl/helm, so arguably it's mostly doing the tasks of the data plane. So it would be great if we have a separate subnet selection option in eks.Cluster & eks.FargateClusters specifically for the kubectl handler Lambda, sth like kubectlSubnets along with kubectlEnvironment.

Having said that, I fully appreciate that you need to address this issue by considering the whole thing holistically. So please feel free to ignore this suggestion if it's contradicting to other factors you might need to consider.

Thx a lot in advance. Jasper