Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more CIDR ranges for AWS backing services #2290

Merged
merged 1 commit into from Mar 31, 2020

Conversation

tlwr
Copy link
Contributor

@tlwr tlwr commented Mar 30, 2020

What

We are starting to run out of IPv4 addresses in production (London) because we only have 3 * /24 which is max 768 (not including reserved)

This PR adds 6 new /24 blocks across AZs

This takes our theoretical per region capacity from 768 single instances to 2304 single instances

Before we could use a CIDR in the security group definition (although it was a bit too big) because 10.0.52.0/22 encapsulated 10.0.{52,53,54}.0/24

However now there are more, there is no one CIDR range which neatly contains all new services, instead we can use the start and the end of the CIDR ranges, as long as they are ordered numerically and we update the correct key in the output variable

Along the way, we can remove zone_index and zone_labels variables which are no longer used

How to review

Code review

Deploy this to your development environment

Ensure that cf security-group elasticache_broker_instances and cf security-group rds_broker_instances are correct

Do something like this for i in $(seq 1 9); do cf create-service postgres tiny-unencrypted-11 test-subnets-$i; sleep 3; done and check that databases can be created in subnets other than 10.0.5{2,3,4}.0/24

Do the same but for Elasticache

Who can review

Not @tlwr

We are starting to run out of IPv4 addresses in production (London)
because we only have 3 * /24 which is max 768 (not including reserved)

Add 6 new /24 blocks across AZs

This takes our theoretical per region capacity from 768 single instances
to 2304 single instances

Before we could use a CIDR in the security group definition (although it
was a bit too big) because 10.0.52.0/22 encapsulated
10.0.{52,53,54}.0/24

However now there are six, there is no one CIDR range which neatly
contains all new services, instead we can use the start and the end of
the cidr ranges, as long as they are ordered numerically and we update
the correct key in the output variable

Along the way, we can remove zone_index and zone_labels variables which
are no longer used

Signed-off-by: Toby Lorne <toby.lornewelch-richards@digital.cabinet-office.gov.uk>
@tlwr
Copy link
Contributor Author

tlwr commented Mar 30, 2020

Please approve but do not merge, as I am waiting for clarification from AWS support

@tlwr
Copy link
Contributor Author

tlwr commented Mar 31, 2020

AWS Support is insisting that silence is the best course of action for customer support

After a lot of googling and some more manual testing I have found:

When you create a new subnet group, take note the number of available IP addresses. If the subnet has very few free IP addresses, you might be constrained as to how many more nodes you can add to the cluster. To resolve this issue, you can assign one or more subnets to a subnet group so that you have a sufficient number of IP addresses in your cluster's Availability Zone. After that, you can add more nodes to your cluster.

From ElastiCache docs

Theoretically RDS should do the same...

@tlwr tlwr merged commit fe118d1 into master Mar 31, 2020
@tlwr tlwr deleted the more-subnets-for-aws-backing-services branch March 31, 2020 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants