Bug: Change in CIDR forces replacement

[MiroslavRepka]

Current Behaviour

Currently, the CIDR for subnets created in VPCs are calculated dynamically. For instance, in the case of AWS, each node pool in the same region has a subnet created based of their position in node pool slice.

{{- range $i, $nodepool := .NodePools }}
resource "aws_subnet" "{{ $nodepool.Name }}_subnet" {
  provider                = aws.k8s_nodepool_{{ $nodepool.Region  }}
  vpc_id                  = aws_vpc.claudie_vpc_{{ $nodepool.Region }}.id
  cidr_block              = "{{ getCIDR "10.0.0.0/24" 2 $i}}"        # This line uses index $i to define the CIDR for the subnet
...

The issue arises, when you change that slice of node pools, i.e. you remove a certain node pool from the input manifest. This then assigns a new CIDR to the subnet, which in the case of AWS forces replacement.

~ cidr_block = "10.0.1.0/24" -> "10.0.0.0/24" # forces replacement

As the subnet is being replaced, all resources that are using subnet ID are being replaced as well, which includes VMs themselves.

~ subnet_id      = "subnet-0ff64c9885134791c" # forces replacement -> (known after apply)

Expected Behaviour

The deletion of a one node pool should not trigger the replacement of another.

Steps To Reproduce

1. Start Claudie
2. Define two-step test case

1.yaml

kubernetes:
  clusters:
    - name: test-cluster
      version: v1.24.0
      network: 192.168.2.0/24
      pools:
        control:
          - oci-control
          - aws-control      # Same providerSpec as aws-compute
        compute:
          - oci-compute
          - aws-compute  # Same providerSpec as aws-control

2.yaml

kubernetes:
  clusters:
    - name: test-cluster
      version: v1.24.0
      network: 192.168.2.0/24
      pools:
        control:
          - oci-control
        compute:
          - oci-compute
          - aws-compute

3. Run the test cases
4. See the replacement of aws-compute in terraformer logs

Anything else to note