AWS VPC endpoint policy for installer-provided VPCs

[wking]

Spun off from #2485. I think we want VPC endpoints in installer-provided VPCs for all services that OpenShift components need (e.g. S3 for the registry, ELB for the cloud provider and ingress operator, etc.). This saves users money, as discussed in #745. It also allows cloud-API access without going through a proxy even from blackholed subnets, as discussed in #2485. Are there any downsides to including these VPC endpoints?

This issues is not discussing VPC endpoints in user-provided VPCs (we can circle back to that once we sort out installer-provided VPCs).

[abhinavdahiya]

This saves users money, as discussed in #745.

s3 api is used for pulling data, so that saves significant money, the rest like ec2, elb don't and there-fore we shouldn't create those.

[wking]

... the rest like ec2, elb don't...

They still save money, just not much, right? "Doesn't save enough money" doesn't seem like enough of a reason to drop them, does it? They still help with the blackholed use case, and they still save some money.

[abhinavdahiya]

They still save money, just not much, right?

AFAIK ec2, elb VPC endpoints don't save money at all..

"Doesn't save enough money" doesn't seem like enough of a reason to drop them, does it?

It's not about "saving money" _ though eveything kinda is ;)_ but more so the s3 VPC endpoint has very clear/ substantial effect to the users, while other don't in a installer created VPC.

They still help with the blackholed use case, and they still save some money.

they don't help in installer created VPC case, and user provided subnets they can't be setup. So not seeing how useful they are...?

[wking]

elb VPC endpoints don't save money at all.

From here:

Data processing charges apply for each Gigabyte processed through the NAT gateway regardless of the traffic’s source or destination

But you'd need a lot of ELB API calls to add up to a significant cost ;).

They still help with the blackholed use case...

they don't help in installer created VPC case...

Ah, fair, you're not going to have an installer-created blackholed use case. You might have a case where a user wanted an installer-created VPC that they twiddled on day two to blackhole the subnets, but they could always add VPC endpoints during that day-2 twiddle.

So remaining point is just "is the small amount of money you save by routing ELB API traffic through a free VPC endpoint instead of a cheap NAT gateway worth the trouble of setting up a new resource?". It's a few lines of Terraform, so not much trouble. I dunno if there's an easy way to actually put a number on the cost savings, but I'm fine tabling this until we think one up.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci-robot]

@openshift-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.