-
Notifications
You must be signed in to change notification settings - Fork 556
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SubnetSpec changes in v1beta2
break existing use-cases
#4026
Comments
/cc @anmolsachan @joeunlog as you both either commented on the PR or raised an issue about this problem |
/triage accepted |
To keep both things happy, what about adding a new entry called subnetMutation and then providing user values there and we would merge that with the existing settings? Though, that might pose a challenge in finding the correct subnet to apply it to. Uh.... |
Just a thought to add to the pile. It's my opinion that CAPA should only offer two methods to get a cluster...
I don't feel there should be a middle ground where CAPA acts on its own to try and reconcile infrastructure based on partial user input. this will lead to bloating the CAPA API just to avoid using an existing tool specialized in solving this problem. To get people who want to graduate from CAPA maintaining all infra to having specified infra the CAPA repo could maintain a sample terraform module to help migrate off the default CAPA infra. @Skarlso the idea of adding subnetMutation won't stop, there will always be something else to mutate and code will fill up with if/else blocks trying to figure out what the user wants and when CAPA assumed wrong the users will just come complaining. CAPA is a consumer of AWS infrastructure, it shouldn't be in the business of creating/maintaining it, but I can concede that method 1 above lowers the barrier to entry to new users and should be maintained. |
To add to this, if you would like to stay in the Kubernetes world but still use terraform together with CAPA, there is an option to use tf-controller. It's a Terraform Kubernetes controller which a lot of options and nice things to reconcile terraform manifests into a cluster. It's great if you are looking for a hybrid solution. |
Not sure I agree with this. The whole purpose of CAPA as far as I see it is to manage AWS infrastructure - creating, updating and deleting it. There is a third (and like more) scenarios where CAPI/CAPA is used besides the two you proposed - there are many platform offerings built on top of it that leverage the standard nature of cluster-api to build a larger product for customers. I think it's also worth pointing out that...
is the second listed feature of CAPA in the readme. If that is was change I think that's a much bigger discussion that would also need to include CAPI and other providers to ensure consistency. |
Sure, but you understand that CAPA cannot cater for every possible need of these platforms. :) I, too, believe that most of CAPA's problems today stem from the fact it's trying to "merge" partial user settings with defaults and stuff that comes from AWS. I wouldn't go as far as to completely rule out user input, but I sure as heck would recommend not touching unmanaged things AT ALL. There are only problems stemming from that route. I even created a proposal to remove settings any kind of tags for unmanaged content. We document everything that we require, then the user can set them up as they like. There are at least 3 issues because of how capa tries to juggle various tags on unmanaged entities. I'm am of the same mind regarding other content, such as subnets. You should be able to have some user input, sure, but at some point, we just have to stop trying to merge partial input with defaults and information coming from AWS. |
I agree with this. If the user is providing their own infrastructure then CAPA should be "read-only". But I do still think (and think you agree if I understood) that when CAPA is managing the infra it should be possible to configure them to some degree. My example of this is being able to set the CIDRs and AWS tags used for subnets. |
We are currently investigating the use of CAPA and this is a huge blocker. The possibility that it can alter resources it doesn't manage with the permission set that is currently required is horrifying. |
In Giant Swarm's fork, I have reverted the buggy changes and added a unit test on top which covers the "managed VPC/subnet" use case. With that change, it seems the functionality is again working as before. See giantswarm#437. We will use that in production in order to get unblocked, and then help drafting the design document for this issue ("Rethinking Networking in CAPA"). |
This issue is labeled with You can:
For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/ /remove-triage accepted |
/priority important-longterm |
/triage accepted |
This issue is labeled with You can:
For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/ /remove-triage accepted |
/triage accepted |
This issue is labeled with You can:
For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/ /remove-triage accepted |
/triage accepted |
This issue is labeled with You can:
For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/ /remove-triage accepted |
We will try covering this issue in a prototype for the CAPA - Rethinking Networking Proposal. See also meeting minutes of CAPA office hours for what was discussed. |
Thanks @AndiDog and @AverageMarcus for creating an alternative proposal. I'm looking through the open issues here and I agree that this is a big regression that shouldn't have happened in the first case. @richardcase and other maintainers, we should prepare for a breaking change and revert the behavior in this case; this was one of the primary values for Cluster API AWS providers. Was the SSA bug originally due to ClusterClass and Managed Topologies? |
/triage accepted |
/kind bug
/kind api-change
The changes introduced in #3748 (aimed at solving server side apply issues with Cluster Class) break existing use-cases around subnets created by CAPA.
The change makes the
ID
property onSubnetSpec
now be a required field where previously this was optional. This change means that configuration to subnets used for CAPA clusters can now only be done if infrastructure is created by the user resulting in CAPA only allowing for either the default subnet layout or a bring-your-own network where the user would be required to create everything (VPC, subnets, etc.) externally before the creation of theAWSCluster
CR.With
v1beta1
and earlier it was possible to specify some of the subnet configuration (such as the CIDR block and the AWS tags) but still have CAPA create the resources for you.For example, we have a situation where we need more control over what resources end up in what subnets and rely on AWS tags and subnet filters to achieve this.
With this approach we can configure CAPA to create all the subnets we need, with the tags we can then filter on, without having to use an external process to create these resources. This is no longer possible with
v1beta2
.In the PR it was acknowledged that this was a breaking change and that we'd need to come up with an alternative to solve this problem going forward but unfortunately was forgotten about so I'm opening this issue to have that discussion and try to come up with a solution to both the original issue the PR was fixing and the use cases currently in use that require configuring subnets in CAPA.
This also fits into the wider discussion around networking in CAPA and the improvements / changes desired.
Related:
Environment:
kubectl version
):/etc/os-release
):The text was updated successfully, but these errors were encountered: