proxy: connect redis with AWS IAM

[khanova]

Problem

Support of IAM Roles for Service Accounts for authentication.

Summary of changes

• Obtain aws 15m-long credentials
• Retrieve redis password from credentials
• Update every 1h to keep connection for more than 12h
• For now allow to have different endpoints for pubsub/stream redis.

TODOs:

• PubSub doesn't support credentials refresh, consider using stream instead.
• We need an AWS role for proxy to be able to connect to both: S3 and elasticache.

Credentials obtaining and connection refresh was tested on xenon preview.

https://github.com/neondatabase/cloud/issues/10365

Checklist before requesting a review

• I have performed a self-review of my code.
• If it is a core feature, I have added thorough tests.
• Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
• If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

• Do not forget to reformat commit message to not include the above checklist

[github-actions]

2706 tests run: 2574 passed, 0 failed, 132 skipped (full report)

---

Code coverage* (full report)

• functions: 28.2% (7208 of 25591 functions)
• lines: 46.7% (44244 of 94675 lines)

* collected from Rust tests only

---

_{The comment gets automatically updated with the latest test results
5d5cc34 at 2024-03-21T13:56:47.891Z :recycle:}

[conradludgate]

Code is rather complex but it seems ok. I don't see anything inherently wrong. How do we plan to roll this out safely?

[khanova]

Code is rather complex but it seems ok. I don't see anything inherently wrong. How do we plan to roll this out safely?

Right now we don't have regional redis anywhere except the preview (and even with the preview it's a bit tricky).

Once we have setup on staging, we could enable it there.

For now I suggest the following plan:

1. Merge
2. Add helm-values
3. Check with the preview in full proxy and cplane setup.