fix(storage): make OCI reauthentication with AWS ECR

[erka]

Few observations about AWS ECR. It looks like there is a big difference between private and public AWS ECR.

Public ECR uses Bearer authorization and returns errors json encoded.

> GET /datadog/datadog HTTP/2
> Host: public.ecr.aws
>
* Request completely sent off
< HTTP/2 401
< content-type: application/json; charset=utf-8
< docker-distribution-api-version: registry/2.0
< www-authenticate: Bearer realm="https://public.ecr.aws/token/",service="public.ecr.aws",scope="aws"
<
{"errors":[{"code":"DENIED","message":"Not Authorized"}]}

The private ECR uses Basic auth and returns errors in plain text.

> GET / HTTP/1.1
> Host: 0.dkr.ecr.us-west-2.amazonaws.com
> Authorization: Basic asdfasfasfd
>
* Request completely sent off
< HTTP/1.1 401 Unauthorized
< Docker-Distribution-Api-Version: registry/2.0
< Www-Authenticate: Basic realm="https://0.dkr.ecr.us-west-2.amazonaws.com/",service="ecr.amazonaws.com"
< Content-Type: text/plain; charset=utf-8
<
Not Authorized

[codecov]

Codecov Report

Attention: Patch coverage is 79.51807% with 17 lines in your changes are missing coverage. Please review.

Project coverage is 72.15%. Comparing base (f997fb9) to head (dd608f4).
Report is 301 commits behind head on main.

Files	Patch %	Lines
internal/oci/ecr/ecr.go	68.29%	8 Missing and 5 partials ⚠️
internal/oci/ecr/credentials_store.go	88.57%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3044      +/-   ##
==========================================
+ Coverage   70.78%   72.15%   +1.36%     
==========================================
  Files          91       99       +8     
  Lines        8729     7502    -1227     
==========================================
- Hits         6179     5413     -766     
+ Misses       2165     1684     -481     
- Partials      385      405      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[erka]

I'm concerned about the complexity of my implementation. In oci/Store func Fetch calls getTarget which creates new oras repo with a new auth client. As the result it requires to cache internal ecr client and cached credential with expiration time. It's hard to understand and hard to maintain.

If anyone has ideas they are welcome here.

[thepabloaguilar]

Sure @erka, I'll take some time to think about an alternative implementation!

[markphelps]

looks good to me! thanks @erka

wdyt @GeorgeMac ?

@thepabloaguilar are you able to build off of this branch to see if it fixes #3043

[GeorgeMac]

Yep, looks great 👏 Nice one @erka