Cut release that includes update to aws-sdk-go dependency in support of IRSA

[chaospuppy]

IRSA support by upgrading aws-sdk-go dependency

The version (v1.15.11) of the aws-sdk-go included in the latest tag of this repository (v2.8.1), is four years old and lacks the functionality required to facilitate IAM Role for Service Accounts (IRSA), which is now the preferred method of retrieving temporary credentials for AWS API calls in a kubernetes context. There was some discussion in reference to doing this on the PR for v2.8.0, but that seems to have died out.

We performed some testing using the edge tag as recommended to rebuild the photon-registry image used by goharbor with a much newer version of aws-sdk-go included in the registry binary, and were successful in leveraging the IRSA resources we have in place to use S3 as a backend for Harbor, whereas prior to the rebuild (i.e. using v2.8.1 of this repository to build the harbor-photon image) we would get errors like this.

In a nutshell, we can confirm that IRSA will work for any project that currently relies on this project for registry, which would be a huge win for a lot of people currently struggling to use S3 for blob storage with registry via IRSA.

[slushysnowman]

This would be a huge win, having to use an IAM user to access S3 storage at the moment, dealing with key rotation etc, is a pain in the ass.

[davidspek]

@milosgajdos Maybe you could give a timeline on this?

[davidspek]

Posting #3809 as this is relevant as well.

[daemon-ian]

Hi @milosgajdos - do you or the team have any update on this?

[milosgajdos]

No immediate plans for this.

[julianandrews]

Any chance we can get this prioritized? Right now, it's impossible to use S3 storage while following current best practices for IAM. This is going to be a stumbling block for most people trying to use S3 with a newly deployed cluster going forward.

If not, at least some mention in the docs that IRSA is not supported would be a real improvement. You can follow the docs exactly right now and if you're depending on IRSA things will just fail. Debugging AWS IAM issues is never fun!

[milosgajdos]

My best recommendation is to look into using the edge tag from https://hub.docker.com/r/distribution/distribution which is the most up-to-date with main branch.

[sirisaacnuketon]

@milosgajdos this doesn't help where we're waiting on it for other projects, like Harbor. Harbor only follow tagged releases. It's also not really great from a perspective of running production workloads, I can't think of many people who'd be happy tagging on edge for anything that needs stability.

What's the reasoning for not releasing this fix?

[milosgajdos]

What's the reasoning for not releasing this fix?

I think I explained it in the last 2.8 release. The TL;DR is: 2.8 is so far behind main that keeping it up to date with main or even close to it is a colossal amount of work, not to mention that it's not even using go modules. We made a decision that 2.8 will only receive security patches from then on and most of the work should focus on v3 release.

I can't speak on behalf of harbor project, I'm afraid. AFAIK it's run by VMware folks and I dont know why they're dependent on the upstream distribution project releases. I agree we need to ramp up the work on the upstream v3 release. There are still a fair amount of items opened in https://github.com/distribution/distribution/milestone/22, though and we're also waiting for the OCI artifact kerfuffle to be sorted.

[chaospuppy]

Perhaps we could take the momentum on this issue and redirect it to getting the Harbor maintainers to pin the base of the goharbor/registry image to a particular digest from the edge tag that has the aws-sdk-go updates included. From cursory testing, Harbor worked fine using a goharbor/registry image based on registry:edge, although I'm sure they'd want more thorough testing that what I already did.

[chaospuppy]

It's not like they're getting more frequent updates by sticking to official releases for registry (not a diss @milosgajdos, I appreciate the explanation 😄 )

[milosgajdos]

I think one of the upstream maintainers is also a harbor maintainer cc: @wy65701436

[milosgajdos]

Closing, addressed in the latest release: https://github.com/distribution/distribution/releases/tag/v3.0.0-alpha.1