Skip to content
This repository has been archived by the owner on May 4, 2021. It is now read-only.

Commit

Permalink
Bump AWS credential helper to latest version (#314)
Browse files Browse the repository at this point in the history 
By bumping the credential helper version we allow user to use the EKS IRSA feature (awslabs/amazon-ecr-credential-helper#183)
  • Loading branch information
@Rowern
Rowern committed Apr 2, 2020
1 parent ae92b34 commit 35b9e70
Show file tree
Hide file tree
Showing 3 changed files with 70 additions and 10 deletions.
54 changes: 49 additions & 5 deletions docs/REGISTRY.md
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Registry configuration

## General info

Makisu supports TLS and Basic Auth with Docker registry (Docker Hub, GCR, and private registries).
By default, TLS is enabled and makisu uses a list of common root CA certs to authenticate registry.

```go
// Config contains Docker registry client configuration.
type Config struct {
Expand All @@ -22,14 +24,17 @@ type Config struct {
```

Configs can be passed in through the `--registry-config` flag, either as filepath, or as a raw json blob :

```
--registry-config='{"gcr.io": {"uber-container-tools/*": {"push_chunk": -1, "security": {"basic": {"username": "_json_key", "password": "<escaped key here>"}}}}}'
```
Consider using the great tool [yq](https://github.com/kislyuk/yq) to convert your yaml configuration into the blob that can be passed in.

Consider using the great tool [yq](https://github.com/kislyuk/yq) to convert your yaml configuration into the blob that can be passed in.

## Examples
For the convenience to work with all public Docker Hub repositories including library/.*, a default config is provided:

For the convenience to work with all public Docker Hub repositories including library/.\*, a default config is provided:

```yaml
index.docker.io:
.*:
Expand All @@ -44,6 +49,7 @@ index.docker.io:
```

Example config for GCR:

```yaml
"gcr.io":
"uber-container-tools/*":
Expand All @@ -52,12 +58,13 @@ Example config for GCR:
basic:
username: _json_key
password: |-
{
<json here>
}
{
<json here>
}
```

To configure your own registry endpoint, pass a custom configuration file to Makisu with `--registry-config=${PATH_TO_CONFIG}`.:

```yaml
[registry]:
[repo]:
Expand All @@ -78,6 +85,7 @@ To configure your own registry endpoint, pass a custom configuration file to Mak
username: <username>
password: <password>
```

Note: For the cert path, you can point to a directory containing your certificates. Makisu will then use all of the certs in that
directory for TLS verification.

Expand All @@ -89,6 +97,7 @@ For ECR, you can export the following [variables](https://docs.aws.amazon.com/cl
If you encounter a certificate validation errors (ex: `x509: certificate signed by unknown authority`) you might want to export the following variable `SSL_CERT_DIR=/makisu-internal/certs/`.

Example AWS ECR config:

```yaml
"someawsregistry":
"my-project/*":
Expand All @@ -98,6 +107,7 @@ Example AWS ECR config:
```

Example GCR config:

```yaml
"gcr.io":
"my-project/*":
Expand All @@ -108,10 +118,26 @@ Example GCR config:

NB: You need to put your config files (ex: aws config/credentials file) inside the /makisu-internal/ dir (and use env variable to specify their locations) in order for the helpers to find and use them when building your images.

### AWS EKS (IAM Roles for Service Accounts - IRSA)

Makisu cleans the environment variables when running so you will need to provide a AWS configuration file.

Example:

```sh
mkdir -p /makisu-internal/.aws/
cp ${AWS_WEB_IDENTITY_TOKEN_FILE} /makisu-internal/.aws/identity_creds
printf "\n[profile eks_role]\nregion = eu-west-3\nrole_arn=${AWS_ROLE_ARN}\nweb_identity_token_file=/makisu-internal/.aws/identity_creds\n" > /makisu-internal/.aws/config
export AWS_SDK_LOAD_CONFIG=true AWS_PROFILE=eks_role AWS_CONFIG_FILE=/makisu-internal/.aws/config SSL_CERT_DIR=/makisu-internal/certs/ SSL_CERT_FILE=/makisu-internal/certs/cacerts.pem
```

Using this example, we are setting an AWS profile (`eks_role`) that will assume the web identity provided to the pod by the EKS service account.

### Using another cred helper

For now makisu handles ECR and GCR as lib instead of calling their binaries.
If you want to use another docker credentials helper, add its binary in the directory `/makisu-internal`, with a name matching `docker-credential-<cred-helper-name>`, then in your configuration:

```yaml
"example.com":
"my-project/*":
Expand All @@ -122,3 +148,21 @@ If you want to use another docker credentials helper, add its binary in the dire
## Handling `BLOB_UPLOAD_INVALID` and `BLOB_UPLOAD_UNKNOWN` errors

If you encounter these errors when pushing your image to a registry, try to use the `push_chunk: -1` option (some registries, despite implementing registry v2 do not support chunked upload, ECR and GCR being one example).

## Handling certificate errors

If you encounter the following error `lstat path: lstat /etc/ssl: no such file or directory`, you should manually specify the SSL cert path. (makisu cleans the env var so the TLS client does not find the correct path if you did overwrite it via the `SSL_CERT_DIR` env var).

Example configuration:

```yaml
"someawsregistry":
"my-project/*":
push_chunk: -1
security:
credsStore: ecr-login
tls:
ca:
cert:
path: /makisu-internal/certs/
```
10 changes: 5 additions & 5 deletions go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ require (
github.com/alicebob/gopher-json v0.0.0-20180125190556-5a6b3ba71ee6 // indirect
github.com/alicebob/miniredis v2.4.5+incompatible
github.com/andres-erbsen/clock v0.0.0-20160526145045-9e14626cd129
github.com/aws/aws-sdk-go v1.19.29 // indirect
github.com/awslabs/amazon-ecr-credential-helper v0.3.0
github.com/aws/aws-sdk-go v1.30.1 // indirect
github.com/awslabs/amazon-ecr-credential-helper v0.4.0
github.com/axw/gocov v0.0.0-20170322000131-3a69a0d2a4ef
github.com/client9/misspell v0.3.4
github.com/docker/distribution v2.7.0+incompatible
Expand All @@ -35,20 +35,20 @@ require (
github.com/onsi/gomega v1.4.3 // indirect
github.com/opencontainers/go-digest v1.0.0-rc1 // indirect
github.com/opencontainers/image-spec v1.0.1 // indirect
github.com/pkg/errors v0.8.0
github.com/pkg/errors v0.9.1
github.com/pressly/chi v3.3.3+incompatible
github.com/prometheus/client_golang v0.9.2 // indirect
github.com/prometheus/common v0.0.0-20181218105931-67670fe90761 // indirect
github.com/sirupsen/logrus v1.4.0 // indirect
github.com/spf13/cobra v0.0.3
github.com/spf13/pflag v1.0.3 // indirect
github.com/stretchr/testify v1.2.2
github.com/stretchr/testify v1.5.1
github.com/yuin/gopher-lua v0.0.0-20181214045814-db9ae37725ec // indirect
go.uber.org/atomic v1.3.2 // indirect
go.uber.org/multierr v1.1.0 // indirect
go.uber.org/zap v1.9.1
golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
golang.org/x/net v0.0.0-20190311183353-d8887717615a
golang.org/x/net v0.0.0-20200202094626-16171245cfb2
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45 // indirect
golang.org/x/tools v0.0.0-20190311215038-5c2858a9cfe5
gopkg.in/yaml.v2 v2.2.2
Expand Down
16 changes: 16 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,15 +12,20 @@ github.com/andres-erbsen/clock v0.0.0-20160526145045-9e14626cd129 h1:MzBOUgng9or
github.com/andres-erbsen/clock v0.0.0-20160526145045-9e14626cd129/go.mod h1:rFgpPQZYZ8vdbc+48xibu8ALc3yeyd64IhHS+PU6Yyg=
github.com/aws/aws-sdk-go v1.19.29 h1:Uusurqi30wm5djEqi3R+9F+TNcGc3aSDSizoPcVp9Sk=
github.com/aws/aws-sdk-go v1.19.29/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.30.1 h1:cUMxtoFvIHhScZgv17tGxw15r6rVKJHR1hsIFRx9hcA=
github.com/aws/aws-sdk-go v1.30.1/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/awslabs/amazon-ecr-credential-helper v0.3.0 h1:AISm4mKRy6QCcymKKAwakyC8Pby/VXTdlA0CSkLyViE=
github.com/awslabs/amazon-ecr-credential-helper v0.3.0/go.mod h1:qdmDatFsfbKXDJD8lc16C4gvMM6Jom09bdHQBPisMJg=
github.com/awslabs/amazon-ecr-credential-helper v0.4.0 h1:LYTmunbYJ8piWElip5hW2NpkEW5JfCbeB9hVHn5LIrc=
github.com/awslabs/amazon-ecr-credential-helper v0.4.0/go.mod h1:qdmDatFsfbKXDJD8lc16C4gvMM6Jom09bdHQBPisMJg=
github.com/axw/gocov v0.0.0-20170322000131-3a69a0d2a4ef h1:kh7Fi8sfEY7aCl42VEEvGv7lez2YCOmO120N1fASWGc=
github.com/axw/gocov v0.0.0-20170322000131-3a69a0d2a4ef/go.mod h1:pc6XrbIn8RLeVSNzXCZKXNst+RTE5Ju/nySYl1Wc0B4=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973 h1:xJ4a3vCFaGF/jqvzLMYoU8P317H5OQ+Via4RmuPwCS0=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI=
github.com/client9/misspell v0.3.4 h1:ta993UF76GwbvJcIo3Y68y/M3WxlpEHPWIGDkJYwzJI=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/docker/distribution v2.7.0+incompatible h1:neUDAlf3wX6Ml4HdqTrbcOHXtfRN0TFIwt6YFL7N9RU=
Expand All @@ -41,6 +46,7 @@ github.com/fsnotify/fsnotify v1.4.7 h1:IXs+QLmnXW2CcXuY+8Mzv/fWEsPGWxqefPtCP5CnV
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/go-redis/redis v6.14.2+incompatible h1:UE9pLhzmWf+xHNmZsoccjXosPicuiNaInPgym8nzfg0=
github.com/go-redis/redis v6.14.2+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
github.com/golang/mock v1.2.0 h1:28o5sBqPkBsMGnC6b4MvE2TzSr5/AT4c/1fLqVGIwlk=
github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0 h1:P3YflyNX/ehuJFLhxviNdFxQPkGK5cDcApsge1SqnvM=
Expand All @@ -57,6 +63,8 @@ github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NH
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af h1:pmfjZENx5imkbgOkpRUYLnmbU7UEFbjtDA2hxJ1ichM=
github.com/jmespath/go-jmespath v0.0.0-20180206201540-c2b33e8439af/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
github.com/jmespath/go-jmespath v0.3.0 h1:OS12ieG61fsCg5+qLJ+SsW9NicxNkg3b25OyT2yCeUc=
github.com/jmespath/go-jmespath v0.3.0/go.mod h1:9QtRXoHjLGCJ5IBSaohpXITPlowMeeYCZ7fLUTSywik=
github.com/juju/ratelimit v1.0.1 h1:+7AIFJVQ0EQgq/K9+0Krm7m530Du7tIz0METWzN0RgY=
github.com/juju/ratelimit v1.0.1/go.mod h1:qapgC/Gy+xNh9UxzV13HGGl/6UXNN+ct+vwSgWNm/qk=
github.com/klauspost/compress v1.4.1 h1:8VMb5+0wMgdBykOV96DwNwKFQ+WTI4pzYURP99CcB9E=
Expand Down Expand Up @@ -84,6 +92,8 @@ github.com/opencontainers/image-spec v1.0.1 h1:JMemWkRwHx4Zj+fVxWoMCFm/8sYGGrUVo
github.com/opencontainers/image-spec v1.0.1/go.mod h1:BtxoFyWECRxE4U/7sNtV5W15zMzWCbyJoFRP3s7yZA0=
github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/pressly/chi v3.3.3+incompatible h1:fc66b0mPg4Dx5Pr86WSsXv0x37dSX6pH0p38GZsvCtU=
Expand All @@ -103,10 +113,13 @@ github.com/spf13/cobra v0.0.3 h1:ZlrZ4XsMRm04Fr5pSFxBgfND2EBVa1nLpiy1stUsX/8=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1 h1:2vfRuCMp5sSVIDSqO8oNnWJq7mPa6KVP3iPIwFBuy8A=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.5.1 h1:nOGnQDM7FYENwehXlg/kFVnos3rEvtKTjRvOWSzb6H4=
github.com/stretchr/testify v1.5.1/go.mod h1:5W2xD1RspED5o8YsWQXVCued0rvSQ+mT+I5cxcmMvtA=
github.com/yuin/gopher-lua v0.0.0-20181214045814-db9ae37725ec h1:vpF8Kxql6/3OvGH4y2SKtpN3WsB17mvJ8f8H1o2vucQ=
github.com/yuin/gopher-lua v0.0.0-20181214045814-db9ae37725ec/go.mod h1:fFiAh+CowNFr0NK5VASokuwKwkbacRmHsVA7Yb1Tqac=
go.uber.org/atomic v1.3.2 h1:2Oa65PReHzfn29GpvgsYwloV9AVFHPDk8tYxt2c2tr4=
Expand All @@ -127,11 +140,14 @@ golang.org/x/net v0.0.0-20181201002055-351d144fa1fc/go.mod h1:mL1N/T3taQHkDXs73r
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a h1:oWX7TPOiFAMXLq8o0ikBYfCJVlRHBcsciT5bXOrH628=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2 h1:CCH4IOTTfewWjGOlSp+zGcjutRKlBEZQ6wTn8ozI/nI=
golang.org/x/net v0.0.0-20200202094626-16171245cfb2/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45 h1:SVwTIAaPC2U/AvvLNZ2a7OVsmBpC8L5BlwK1whH3hm0=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f h1:Bl/8QSvNqXvPGPGXa2z5xUTmV7VDcZyvRZ+QQXkXTZQ=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4 h1:YUO/7uOKsKeq9UokNS62b8FYywz3ker1l1vDZRCRefw=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e h1:o3PsSEY8E4eXWkXrIP9YJALUkVZqzHJT5DOasTyn8Vs=
Expand Down

0 comments on commit 35b9e70

Please sign in to comment.