Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can use existing vpc and subnets #42

Closed
tigerwings opened this issue Jun 6, 2018 · 41 comments · Fixed by #305

Comments

@tigerwings
Copy link

commented Jun 6, 2018

need the capability to specify vpc and subnets in create cluster command to reuse the existing direct connect vpc.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jun 6, 2018

I think there will be two major modes here:

  1. use a preconfigured VPC, but create new subnets (easiest for the user who is simply short of VPCs)
  2. use a preconfigured VPC and subnets (someone who wants certain kind of configures, e.g. as mention above - with direct connect was; or more generally someone who has to control networking separately)

I think it must be possible to go with first option and and instruct the user to configure routes and additional security features separately.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jun 6, 2018

@tigerwings I have a specific question to you personally – can you use VPC peering to connect your EKS VPC with an existing VPC and enable direct connect that way?

@tigerwings

This comment has been minimized.

Copy link
Author

commented Jun 6, 2018

@errordeveloper , vpc peering is not a valid solution. The main reason to use DC VPCs is that we need to access our internal service from DC VPCs. On the other hand, we may setup and tear down an EKS cluster often, the peering ops have too much overheads.

@archisgore

This comment has been minimized.

Copy link
Contributor

commented Jun 27, 2018

I'm working on this issue as we speak (if nobody else already is). This is a scenario I need for myself in prod. :-)

In I only found eksctl because I was about to write a shell script to rinse-repeat launch clusters. So I'll get around to this very soon.

@baank

This comment has been minimized.

Copy link

commented Jul 9, 2018

@archisgore .. Any status update on this ? We can look into picking this up also.

If you are wondering of a use case here's one. In many large enterprises e.g. ours we centrally control the core networking e.g. VPCs but then freely allow various teams to pick and choose what services they want to use. We are wanting to use eksctl but are strictly forbidden from creating VPCs.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jul 9, 2018

@baank thanks for describing your use-case, it's very helpful! Could you clarify if you havr to use pre-existing subnets also, or it would be okay for eksctl to create dedicated subnets?

@baank

This comment has been minimized.

Copy link

commented Jul 9, 2018

@errordeveloper .. No we are not allowed to touch the subnets either.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jul 9, 2018

@baank so you would need two subnets. In your case, would you be able to get two subnets that are dedicated for use with EKS or you would have to use subnets that are also used for other things? If there would be other things, are those gonna be just any AWS workloads or another Kubernetes clusters or something?

What size of subnets would you be able to provide? Here is what we currently use with the dedicated VPC (based on standard CloudFormation template provided):

VpcBlock: 192.168.0.0/16
Subnet01Block: 192.168.64.0/18
Subnet02Block: 192.168.128.0/18

I'd have to check if anything smaller would work, I don't know right now.

Also, are you able to create EC2 security groups or it's a similar situation there?

@baank

This comment has been minimized.

Copy link

commented Jul 9, 2018

We have a number of subnets but they are in different VPCs and AZs:

subnet-A -> vpc1-data-a -> ap-southeast-2a
subnet-B -> vpc1-data-b -> ap-southeast-2b
subnet-C -> vpc1-application-a -> ap-southeast-2a
subnet-D -> vpc1-application-b -> ap-southeast-2b

The subnets are:
x.x.x.x/24
x.x.x.x/25
x.x.x.x/26

And the subnets are basically a free for all. Each team can decide what they want and unless we run out of IP addresses (sometimes happens) then no one particular cares. Some have Kube-AWS clusters, EC2 instances etc. Just picture hundreds of developers each treating it as though it was there personal AWS account.

And no one really cares too much about security groups since they only restrict internal traffic within the VPCs and there are probably easier ways for a rogue employee to steal data or hack into systems.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jul 10, 2018

@baank thanks again, this is very helpful for us! Please note that at the moment EKS is not available in ap-southeast-2, I am not sure what is the timeline.

@nukepuppy

This comment has been minimized.

Copy link

commented Sep 4, 2018

just checking in.. as defining the CIDR range is probably one of the most basic things folks will need to support..

Though this issue is about re-using existing? Is there a way for new cluster creation to use the CIDR of choosing? - if not where is that on roadmap?

this blocks any kind of peering or direct connect setups which allow for restricted resources elsewhere.. and well.. that's pretty essential

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Sep 4, 2018

@nukepuppy thanks for clarifying the use-case, could you please open another issue, unless using a pre-existing VPC and subnets would be satisfactory (or even somewhat more beneficial) for you? We are planning to start working on this issue soon. I was hoping subnet CIDR params could wait until we have Cluster API support and can expose those and other advanced parameters via a config file and avoid exploding the number of CLI flags, if we can. If this is a burning issue, please consider looking at the code and opening a PR, I am happy to help you along.

@nukepuppy

This comment has been minimized.

Copy link

commented Sep 5, 2018

@errordeveloper i'll open a new issue.. The use case is net new environments / VPCs which can be used in an enterprise environment where IPAM is enforced etc.. and using the standard 192 or 172 class Bs wont do.. so yes I'll definitely open a new issue.

As for playing with a PR .. if time allows I'm definitely down to dive into it

@darrenhaken

This comment has been minimized.

Copy link

commented Sep 7, 2018

How's the progress coming with this? I also have the need to use an existing VPC and subnets.

@richardcase

This comment has been minimized.

Copy link
Collaborator

commented Sep 7, 2018

@darrenhaken - if we supported --vpc and --subnets (like kops) on create cluster would that satisfy your requirement?

@darrenhaken

This comment has been minimized.

Copy link

commented Sep 7, 2018

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Sep 7, 2018

@darrenhaken

This comment has been minimized.

Copy link

commented Sep 7, 2018

I'm basing the best option around how Terraform does it:

resource "aws_eks_cluster" "demo" {
  name            = "${var.cluster-name}"
  role_arn        = "${aws_iam_role.demo-cluster.arn}"

  vpc_config {
    security_group_ids = ["${aws_security_group.demo-cluster.id}"]
    subnet_ids         = ["${aws_subnet.demo.*.id}"]
  }

  depends_on = [
    "aws_iam_role_policy_attachment.demo-cluster-AmazonEKSClusterPolicy",
    "aws_iam_role_policy_attachment.demo-cluster-AmazonEKSServicePolicy",
  ]
}

Here Terraform allows passing the VPC and the subnet IDs

@richardcase

This comment has been minimized.

Copy link
Collaborator

commented Sep 7, 2018

I'll take a stab at this next week if no one picks it up.

@darrenhaken

This comment has been minimized.

Copy link

commented Sep 7, 2018

I think the security groups being args is also good

@mduarte

This comment has been minimized.

Copy link

commented Sep 7, 2018

We also have a use case where we would like to have the iam-role as a parameter as well.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Sep 7, 2018

@darrenhaken

This comment has been minimized.

Copy link

commented Sep 7, 2018

@darrenhaken

This comment has been minimized.

Copy link

commented Oct 15, 2018

Any update on how this is going?

@richardcase

This comment has been minimized.

Copy link
Collaborator

commented Oct 15, 2018

@darrenhaken - i didn't get around to starting it. I'm going to try and get scaling finished and then if no one else picks it i will add this as my next task this week.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Oct 15, 2018

I think we might want to create a separate issue with a design proposal and discuss details there.

At the moment, I am thinking of the following 3 modes being supported eventually:

  1. dedicated VPC (what we currently have already)
  2. --subnets specified by the user, VPC ID determined from that (which implies 3 subnets must be given, and routing table as well as any other resources are already configured, subnet size and availably of allocatable IP addresses is up to the user to take care of; should be possible to use default VPC)
  3. --vpc specified by the user, all subnets are created (should be possible to use default VPC)

We can start by adding 2 first, as that seems like the least common denominator.

As mentioned earlier, it'd be good also provide a utility command that create pre-requisite VPC stack, so that could be used in the case when VPC management is delegated to a network operations teams who may prefer to use eksctl for convenience, such utility may also write out CloudFormation template that can be customised to enable VPC peering or any other advanced use-cases.


I'm happy to use the above as starting point for a proposal, but please let me know if there is anything I've missed, before I go ahead and create one.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Oct 27, 2018

At this point --subnets flag (as described above) would be very easy to implement, perhaps even --vpc flag. For next week's release I'd like to focus on #287 and #281, but would be keen to review a PR for this. Any takers? It'd mostly a matter of adding a flag and making sure plumbing is right.

@ArseniiPetrovich

This comment has been minimized.

Copy link

commented Nov 2, 2018

+1 to this issue. At the moment we are in the process of developing a script for internal use that uses eksctl as one of the core features to create EKS cluster. Possibility to specify an existing VPC is a great feature to add.
Is there any chance this one would be finished before the end of november, please?)

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Nov 2, 2018

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Nov 5, 2018

We have a design proposal now – #303, I'd love to hear feedback from folks here. I am also intending to add --vpc-private-subnets and --vpc-public-subnets in this weeks release, we can discuss and amend the proposal as needed later on also.

@genums

This comment has been minimized.

Copy link

commented Nov 6, 2018

Hi,

Will the new changes be available somewhere for us to link to or update the EKS quick start? Currently, the setup in the quick start doesn't work because of our need to use our existing VPC and public subnets which routes the path to the internet through our data centers. Your proposal above would cure a lot of headaches.

Thank you

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Nov 6, 2018

eksctl triage board automation moved this from v0.1.x to Done Nov 7, 2018

@errordeveloper errordeveloper referenced this issue Nov 7, 2018
2 of 3 tasks complete
@darrenhaken

This comment has been minimized.

Copy link

commented Nov 7, 2018

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Nov 7, 2018

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Nov 8, 2018

@ORESoftware

This comment has been minimized.

Copy link

commented Jun 13, 2019

Anybody know how to reuse/reference an existing VPC?

eksctl create cluster --vpc=vpc-0f9e518dea1b2c523 

?

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jun 13, 2019

@ORESoftware

This comment has been minimized.

Copy link

commented Jun 13, 2019

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jun 14, 2019

its really not well documented, and it seems like a popular use case

Could you please tell me what is missing here?

on that same subject, its unclear in the cluster UI which are private and which are public subnets, I assume by default they are all public?

Do you mean the EKS console?

@ORESoftware

This comment has been minimized.

Copy link

commented Jun 14, 2019

yeah in the EKS console, it's unclear whether subnets are public/private (is it more clear somewhere else, idk?)

Anyway, for long readme's, it would be nice to have a table of contents (ToC) at the top and link to the subsections.

@errordeveloper

This comment has been minimized.

Copy link
Member

commented Jun 17, 2019

yeah in the EKS console, it's unclear whether subnets are public/private (is it more clear somewhere else, idk?)

The subnets that eksctl creates for you will have it in the name, but it's up to you to name the ones that you are managing yourself.

it would be nice to have a table of contents

We are reworking documentation at the moment, new website is coming any day now! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.