Support Public IPv4 Pool for resources created with EIP

[mtulio]

/kind feature

Describe the solution you'd like
[A clear and concise description of what you want to happen.]

• As a CAPA user, I would like to decrease the impact of additional cost of Public IPv4 charges (introduced by AWS in February 2024) in my CI cluster pool consuming thousands clusters/day from Amazon-provided IPv4 pool (default), so I can keep the total cost of my fleet under control.
• As a CAPA user, I would like to be able to deploy clusters consuming public IPv4 from my custom [IPv4 pool that I brought to AWS][byoip], so I can have control of public IPs used by my clusters.

I would like to ask if we could, optionally, provide support of providing a Public IPv4 Pool, allowing the resources created with Elastic IP (EIP) consuming preferably from that pool instead of Amazon-provided public IPv4.

Currently, CAPA consumes public IP address when:

• NAT Gateways are created, one EIP for each zone
• the API Network Load Balancer is public, one EIP for each public subnet
• one by machine is deployed in public subnets

Giving the example of a cluster deployed in us-east-1, when using all 6 zones, the additional fees applied for each cluster is something like (using one bastion/bootstrap node with public IP): (6 + 6 + 1) * $3.6 = $46.8

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Open Questions:

• Considering the pool can be exhausted (no more Public IPv4 to allocated) and Public IPv4 Pool API provides the amount of IP allocated, can we provide an flag or option to allow fallback to Amazon-provided when the BYOIP pool is exhausted?

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPA/CAPI contributors determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.