Build NAT gateways in parallel

[randomvariable]

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]
Combination of #1752 and lack of parallel NAT gateway creation increases the cluster creation time significantly, failing e2e tests. #1789 works around this, but we should be testing multi-AZ anyway.

Having NAT gateways build in parallel or otherwise having the controller return and reconcile would speed up cluster creation sufficiently.

What did you expect to happen:

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]

Environment:

• Cluster-api-provider-aws version:
• Kubernetes version: (use kubectl version):
• OS (e.g. from /etc/os-release):

[detiber]

I don't necessarily know if building them in parallel is what we need, or refactoring to create all needed NAT Gateways, Subnets, and other resources that we need multiple of during the course of reconciliation prior to any waiting that is done.

Either way +1 to the idea of minimizing the wait time needed for creating multiple instances of a type of resource without increasing the reconciliation time linearly with the number of instances wherever we can.

[randomvariable]

or refactoring to create all needed NAT Gateways, Subnets, and other resources that we need multiple of during the course of reconciliation prior to any waiting that is done.

Yup, I meant that the effect would be parallel NAT gateway creation, thanks for clarifying!

[gab-satchi]

/assign
/lifecycle active

[randomvariable]

Maybe worth holding on #1752, as it will likely conflict?