Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can use existing vpc and subnets #42

Closed
tigerwings opened this issue Jun 6, 2018 · 41 comments · Fixed by #305
Closed

Can use existing vpc and subnets #42

tigerwings opened this issue Jun 6, 2018 · 41 comments · Fixed by #305
Labels
help wanted Extra attention is needed kind/feature New feature or request
Milestone
0.1.x – improvements

Comments

@tigerwings
Copy link

need the capability to specify vpc and subnets in create cluster command to reuse the existing direct connect vpc.
@errordeveloper
Copy link
Contributor

I think there will be two major modes here:

  1. use a preconfigured VPC, but create new subnets (easiest for the user who is simply short of VPCs)
  2. use a preconfigured VPC and subnets (someone who wants certain kind of configures, e.g. as mention above - with direct connect was; or more generally someone who has to control networking separately)

I think it must be possible to go with first option and and instruct the user to configure routes and additional security features separately.

@errordeveloper
Copy link
Contributor

@tigerwings I have a specific question to you personally – can you use VPC peering to connect your EKS VPC with an existing VPC and enable direct connect that way?

@errordeveloper errordeveloper added kind/feature New feature or request help wanted Extra attention is needed labels Jun 6, 2018
@tigerwings
Copy link
Author

@errordeveloper , vpc peering is not a valid solution. The main reason to use DC VPCs is that we need to access our internal service from DC VPCs. On the other hand, we may setup and tear down an EKS cluster often, the peering ops have too much overheads.

@errordeveloper errordeveloper mentioned this issue Jun 21, 2018
Closed
@archisgore
Copy link
Contributor

I'm working on this issue as we speak (if nobody else already is). This is a scenario I need for myself in prod. :-)

In I only found eksctl because I was about to write a shell script to rinse-repeat launch clusters. So I'll get around to this very soon.

@errordeveloper errordeveloper mentioned this issue Jul 5, 2018
Closed
@nadenf
Copy link

nadenf commented Jul 9, 2018 •
edited
Loading

@archisgore .. Any status update on this ? We can look into picking this up also.

If you are wondering of a use case here's one. In many large enterprises e.g. ours we centrally control the core networking e.g. VPCs but then freely allow various teams to pick and choose what services they want to use. We are wanting to use eksctl but are strictly forbidden from creating VPCs.

@errordeveloper
Copy link
Contributor

@baank thanks for describing your use-case, it's very helpful! Could you clarify if you havr to use pre-existing subnets also, or it would be okay for eksctl to create dedicated subnets?

@nadenf
Copy link

nadenf commented Jul 9, 2018

@errordeveloper .. No we are not allowed to touch the subnets either.

@errordeveloper
Copy link
Contributor

errordeveloper commented Jul 9, 2018 •
edited
Loading

@baank so you would need two subnets. In your case, would you be able to get two subnets that are dedicated for use with EKS or you would have to use subnets that are also used for other things? If there would be other things, are those gonna be just any AWS workloads or another Kubernetes clusters or something?

What size of subnets would you be able to provide? Here is what we currently use with the dedicated VPC (based on standard CloudFormation template provided):

VpcBlock: 192.168.0.0/16
Subnet01Block: 192.168.64.0/18
Subnet02Block: 192.168.128.0/18

I'd have to check if anything smaller would work, I don't know right now.

Also, are you able to create EC2 security groups or it's a similar situation there?

@nadenf
Copy link

nadenf commented Jul 9, 2018

We have a number of subnets but they are in different VPCs and AZs:

subnet-A -> vpc1-data-a -> ap-southeast-2a
subnet-B -> vpc1-data-b -> ap-southeast-2b
subnet-C -> vpc1-application-a -> ap-southeast-2a
subnet-D -> vpc1-application-b -> ap-southeast-2b

The subnets are:
x.x.x.x/24
x.x.x.x/25
x.x.x.x/26

And the subnets are basically a free for all. Each team can decide what they want and unless we run out of IP addresses (sometimes happens) then no one particular cares. Some have Kube-AWS clusters, EC2 instances etc. Just picture hundreds of developers each treating it as though it was there personal AWS account.

And no one really cares too much about security groups since they only restrict internal traffic within the VPCs and there are probably easier ways for a rogue employee to steal data or hack into systems.

@errordeveloper
Copy link
Contributor

@baank thanks again, this is very helpful for us! Please note that at the moment EKS is not available in ap-southeast-2, I am not sure what is the timeline.

@errordeveloper errordeveloper mentioned this issue Jul 18, 2018
Closed
@errordeveloper errordeveloper added this to the 0.1.x – improvements milestone Jul 23, 2018
@nukepuppy
Copy link

just checking in.. as defining the CIDR range is probably one of the most basic things folks will need to support..

Though this issue is about re-using existing? Is there a way for new cluster creation to use the CIDR of choosing? - if not where is that on roadmap?

this blocks any kind of peering or direct connect setups which allow for restricted resources elsewhere.. and well.. that's pretty essential

@errordeveloper
Copy link
Contributor

@nukepuppy thanks for clarifying the use-case, could you please open another issue, unless using a pre-existing VPC and subnets would be satisfactory (or even somewhat more beneficial) for you? We are planning to start working on this issue soon. I was hoping subnet CIDR params could wait until we have Cluster API support and can expose those and other advanced parameters via a config file and avoid exploding the number of CLI flags, if we can. If this is a burning issue, please consider looking at the code and opening a PR, I am happy to help you along.

@nukepuppy
Copy link

@errordeveloper i'll open a new issue.. The use case is net new environments / VPCs which can be used in an enterprise environment where IPAM is enforced etc.. and using the standard 192 or 172 class Bs wont do.. so yes I'll definitely open a new issue.

As for playing with a PR .. if time allows I'm definitely down to dive into it

@darrenhaken
Copy link

How's the progress coming with this? I also have the need to use an existing VPC and subnets.

@richardcase
Copy link
Contributor

@darrenhaken - if we supported --vpc and --subnets (like kops) on create cluster would that satisfy your requirement?

@darrenhaken
Copy link

darrenhaken commented Sep 7, 2018 via email

@errordeveloper
Copy link
Contributor

errordeveloper commented Sep 7, 2018 via email •
edited
Loading

@darrenhaken
Copy link

I'm basing the best option around how Terraform does it:

resource "aws_eks_cluster" "demo" {
  name            = "${var.cluster-name}"
  role_arn        = "${aws_iam_role.demo-cluster.arn}"

  vpc_config {
    security_group_ids = ["${aws_security_group.demo-cluster.id}"]
    subnet_ids         = ["${aws_subnet.demo.*.id}"]
  }

  depends_on = [
    "aws_iam_role_policy_attachment.demo-cluster-AmazonEKSClusterPolicy",
    "aws_iam_role_policy_attachment.demo-cluster-AmazonEKSServicePolicy",
  ]
}

Here Terraform allows passing the VPC and the subnet IDs

@richardcase
Copy link
Contributor

I'll take a stab at this next week if no one picks it up.

@darrenhaken
Copy link

I think the security groups being args is also good

@mduarte
Copy link

mduarte commented Sep 7, 2018

We also have a use case where we would like to have the iam-role as a parameter as well.

@errordeveloper
Copy link
Contributor

errordeveloper commented Sep 7, 2018 via email

@darrenhaken
Copy link

darrenhaken commented Sep 7, 2018 via email

@darrenhaken
Copy link

Any update on how this is going?

@richardcase
Copy link
Contributor

@darrenhaken - i didn't get around to starting it. I'm going to try and get scaling finished and then if no one else picks it i will add this as my next task this week.

@errordeveloper
Copy link
Contributor

errordeveloper commented Oct 15, 2018 •
edited
Loading

I think we might want to create a separate issue with a design proposal and discuss details there.

At the moment, I am thinking of the following 3 modes being supported eventually:

  1. dedicated VPC (what we currently have already)
  2. --subnets specified by the user, VPC ID determined from that (which implies 3 subnets must be given, and routing table as well as any other resources are already configured, subnet size and availably of allocatable IP addresses is up to the user to take care of; should be possible to use default VPC)
  3. --vpc specified by the user, all subnets are created (should be possible to use default VPC)

We can start by adding 2 first, as that seems like the least common denominator.

As mentioned earlier, it'd be good also provide a utility command that create pre-requisite VPC stack, so that could be used in the case when VPC management is delegated to a network operations teams who may prefer to use eksctl for convenience, such utility may also write out CloudFormation template that can be customised to enable VPC peering or any other advanced use-cases.

I'm happy to use the above as starting point for a proposal, but please let me know if there is anything I've missed, before I go ahead and create one.

@errordeveloper errordeveloper mentioned this issue Oct 19, 2018
Merged
3 tasks
@nukepuppy nukepuppy mentioned this issue Oct 23, 2018
Closed
@errordeveloper errordeveloper mentioned this issue Oct 24, 2018
Merged
2 tasks
@errordeveloper
Copy link
Contributor

errordeveloper commented Oct 27, 2018 •
edited
Loading

At this point --subnets flag (as described above) would be very easy to implement, perhaps even --vpc flag. For next week's release I'd like to focus on #287 and #281, but would be keen to review a PR for this. Any takers? It'd mostly a matter of adding a flag and making sure plumbing is right.

@ArseniiPetrovich
Copy link

+1 to this issue. At the moment we are in the process of developing a script for internal use that uses eksctl as one of the core features to create EKS cluster. Possibility to specify an existing VPC is a great feature to add.
Is there any chance this one would be finished before the end of november, please?)

@errordeveloper
Copy link
Contributor

errordeveloper commented Nov 2, 2018 via email

@errordeveloper
Copy link
Contributor

We have a design proposal now – #303, I'd love to hear feedback from folks here. I am also intending to add --vpc-private-subnets and --vpc-public-subnets in this weeks release, we can discuss and amend the proposal as needed later on also.

@genums
Copy link

genums commented Nov 6, 2018

Hi,

Will the new changes be available somewhere for us to link to or update the EKS quick start? Currently, the setup in the quick start doesn't work because of our need to use our existing VPC and public subnets which routes the path to the internet through our data centers. Your proposal above would cure a lot of headaches.

Thank you

@errordeveloper
Copy link
Contributor

errordeveloper commented Nov 6, 2018 via email •
edited
Loading

@errordeveloper errordeveloper mentioned this issue Nov 7, 2018
Merged
3 tasks
@darrenhaken
Copy link

darrenhaken commented Nov 7, 2018 via email

@errordeveloper
Copy link
Contributor

errordeveloper commented Nov 7, 2018 via email

@errordeveloper
Copy link
Contributor

errordeveloper commented Nov 8, 2018 via email

@ORESoftware
Copy link

Anybody know how to reuse/reference an existing VPC?

eksctl create cluster --vpc=vpc-0f9e518dea1b2c523

?

@errordeveloper
Copy link
Contributor

errordeveloper commented Jun 13, 2019 via email

@ORESoftware
Copy link

ORESoftware commented Jun 13, 2019 via email •
edited
Loading

@errordeveloper
Copy link
Contributor

its really not well documented, and it seems like a popular use case

Could you please tell me what is missing here?

on that same subject, its unclear in the cluster UI which are private and which are public subnets, I assume by default they are all public?

Do you mean the EKS console?

@ORESoftware
Copy link

ORESoftware commented Jun 14, 2019 •
edited
Loading

yeah in the EKS console, it's unclear whether subnets are public/private (is it more clear somewhere else, idk?)

Anyway, for long readme's, it would be nice to have a table of contents (ToC) at the top and link to the subsections.

@errordeveloper
Copy link
Contributor

yeah in the EKS console, it's unclear whether subnets are public/private (is it more clear somewhere else, idk?)

The subnets that eksctl creates for you will have it in the name, but it's up to you to name the ones that you are managing yourself.

it would be nice to have a table of contents

We are reworking documentation at the moment, new website is coming any day now! :)

torredil pushed a commit to torredil/eksctl that referenced this issue May 20, 2022
@k8s-ci-robot
Merge pull request eksctl-io#42 from leakingtapan/topology
fa25e65 
Implement topology awareness support for dynamic provisioning
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed kind/feature New feature or request
Projects
None yet
Milestone
0.1.x – improvements
Development

Successfully merging a pull request may close this issue.

Enable use of existing subnets eksctl-io/eksctl