Unable to --publish to private registry on Windows

[naavo]

We are trying to use pack cli in one of use cases and ran into this

Pack: v0.0.9 - Windows
Use Case: Trying to wrap a spring boot application into a image and push it onto a private protected registry (i.e need access to do a push)

Issue: Unable to push the built image onto the registry using the --publish option because of authorization issues. Though we are able to push / pull images onto the same registry using simple docker push / pull commands from command line.

We used the command

pack build <repo/image> --publish

The related bit of logging that we see are as follows

�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:18 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:18 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:20 adding layer 'app' with diffID 'sha256:0f212386e9a49e793ea6765522541a8f6e4642642c7e5632be7eee4f73d36527'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:20 adding layer 'config' with diffID 'sha256:1b60f40720025082c996bd368c517a1ce6a527191c1fbac49447db8993024c99'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:21 adding layer 'launcher' with diffID 'sha256:d77dc7ed6207d6bb9c389aa5f087ea7fffea9238e2de84b03f8b3c1152e1e58f'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:22 adding layer 'io.buildpacks.samples.java:jdk' with diffID 'sha256:f68db36d1f3a82c357c356c44db92bfd89d9ff2bbe63efd3f9d0efd824cf2414'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 setting metadata label 'io.buildpacks.lifecycle.metadata'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 setting env var 'PACK_LAYERS_DIR=/workspace'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 setting env var 'PACK_APP_DIR=/workspace/app'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 setting entrypoint '/lifecycle/launcher'
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 setting empty cmd
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 writing image
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:25 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:7308e914506c09fa6d2242368545f55462e024f785e21b21b3e90403081a9336
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:705054bc3f5bd722eb8f026532447fac897c521d8906eb36e79a60d0fb0606fa
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:38e2e6cd5626f31cea1a0a5751a9a7e6564c589a3388dcf84f00d4bb98146844
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:09b3ec4f3bede08b94d415ed2318e8766fb189bf31e69c9ed5d5db5055982f07
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:4c51b7d9e9ea44f5e1a432b04fd1d3a2accbcc8e8c2ba7455155a6661c868fb8
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:2501aaff82c2d492dc3f4fd4f526fd3c2e7fd090297559566733f170d92d692f
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:01e734c850498f929b8fb427f845545d3ebf3165afdb77c85a9f7dcbda1adc60
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:c7051e0695642c5bf01467a4a64106cabfa20e57fee91b31ee73f7308bccf5bd
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:80e339c0631fb56471f967a9838454f62212fcf097b3424152b3a2cd266f52aa
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 existing blob: sha256:2da8bfb3b5b741820e6cb29a71f66e8d944ddad2d00c6dc1e31461120ed39e46
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26
�[90m�[0m[�[36mexporter�[0m] *** Image: <REPO/IMAGE>
�[90m�[0m[�[36mexporter�[0m] 2019/03/11 19:33:26 Error: failed to : UNAUTHORIZED: The client does not have permission to push to the repository.; map[]
�[90m�[0m�[31;1mERROR: �[0mrun lifecycle/exporter: failed with status code: 7

[trisberg]

We are seeing similar failures in riff on Windows when trying to push to Docker Hub:

�[90m�[0m�[36m===> EXPORTING�[0m
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:22 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:23 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:23 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:23 reusing layer 'app' with diffID 'sha256:9570aa8044f8ea24c557472d8739535cca1e019f4b401466e069ee147edfc5c8'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:24 reusing layer 'config' with diffID 'sha256:7a28859b4948ec10654c3cd6cec0597acf215e9c743c842562ff504a0b395a40'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:24 reusing layer 'launcher' with diffID 'sha256:d77dc7ed6207d6bb9c389aa5f087ea7fffea9238e2de84b03f8b3c1152e1e58f'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:24 reusing layer 'io.buildpacks.samples.nodejs:node_modules' with diffID 'sha256:302618f06239200810a871ffc701a367cf9ce718838c021cd5c7e11d881f29b8'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 reusing layer 'io.buildpacks.samples.nodejs:nodejs' with diffID 'sha256:b4bdbfc67313d26a53a90c4bd1783455cbef427518d3158e52a76cd06da108f0'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 setting metadata label 'io.buildpacks.lifecycle.metadata'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 setting env var 'PACK_LAYERS_DIR=/workspace'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 setting env var 'PACK_APP_DIR=/workspace/app'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 setting entrypoint '/lifecycle/launcher'
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 setting empty cmd
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 writing image
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 Unable to read "/home/pack/.docker/config.json": open /home/pack/.docker/config.json: no such file or directory
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:4c51b7d9e9ea44f5e1a432b04fd1d3a2accbcc8e8c2ba7455155a6661c868fb8
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:fd1e7cb6854b66840d8c274bc5119fee238342d860a6caa4041988895ba8f106
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:2da8bfb3b5b741820e6cb29a71f66e8d944ddad2d00c6dc1e31461120ed39e46
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:b04e5e8589629745096938315a58ecb6a2bf60c08e301ce500d7ac696e175e77
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:c7051e0695642c5bf01467a4a64106cabfa20e57fee91b31ee73f7308bccf5bd
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:2501aaff82c2d492dc3f4fd4f526fd3c2e7fd090297559566733f170d92d692f
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:09b3ec4f3bede08b94d415ed2318e8766fb189bf31e69c9ed5d5db5055982f07
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:7308e914506c09fa6d2242368545f55462e024f785e21b21b3e90403081a9336
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:0a3e3e90916025a7376d1a69085e2b905fb953550121a8e9b5c89bbbc8a9a750
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:705054bc3f5bd722eb8f026532447fac897c521d8906eb36e79a60d0fb0606fa
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:6abeded1f3b6f90b053ddd7f5e51675dcd7aa8444b41696f550af7bbc6514135
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:38e2e6cd5626f31cea1a0a5751a9a7e6564c589a3388dcf84f00d4bb98146844
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:a0c8d2ac918499f2b98a5b7698228f05488a1fb8b9630a81b2692bce7decc1b1
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:01e734c850498f929b8fb427f845545d3ebf3165afdb77c85a9f7dcbda1adc60
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 existing blob: sha256:80e339c0631fb56471f967a9838454f62212fcf097b3424152b3a2cd266f52aa
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25
�[90m�[0m[�[36mexporter�[0m] *** Image: trisberg/node-app:pack@
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:56:25 Error: failed to : UNAUTHORIZED: authentication required; [map[Name:trisberg/node-app Action:pull Type:repository Class:] map[Action:push Type:repository Class: Name:trisberg/node-app]]
�[90m�[0m�[31;1mERROR: �[0mrun lifecycle/exporter: failed with status code: 7

Also getting error trying to use gcr.io:

�[90m�[0m�[36m===> EXPORTING�[0m
�[90m�[0m[�[36mexporter�[0m] 2019/03/25 18:52:43 Error: connect to repo store 'packs/run:v3alpha2': <empty transport.Error response>
�[90m�[0m�[31;1mERROR: �[0mrun lifecycle/exporter: failed with status code: 1

[ekcasey]

thanks for the report @naavo and the additional context @trisberg. We will look at this before we release the beta of pack

[ekcasey]

@naavo are you seeing this problem with Docker Hub specifically or also with other registries?

@djoyahoy and I investigated and we found a problem publishing to Docker Hub. The issue as we understand it boils down to a problem with the interaction between google/go-containerregistry & docker-credential-wincred.

When you docker login on windows your auth in stored in wincred and associated with the registry https://index.docker.io/v1/. pack uses the google/go-containerregistry (ggcr) library to resolve the correct auth for a registry. When your registry is Docker Hub ggcr asks the credential helper for the auth for https://index.docker.io instead of https://index.docker.io/v1/. docker-credential-wincred then reports that there is no auth for that registry.

It seems like docker-credential-osxkeychain is smart enough to know thathttps://index.docker.io/v1/ and https://index.docker.io are the same, but docker-credential-wincred is not.

We can add a hack to pack to fix this but it would probably be best to PR changes upstream.

As a workaround you could add your auth directly to the docker config.json. Plain text credentials aren't great so proceed at your own risk. Alternatively you could directly store auth for https://index.docker.io using docker-credential-wincred (we haven't tried either of these workarounds yet, but they should work).

[naavo]

@ekcasey thank you for the explanation. We were trying to push to a private enterprise registry on artifactory.

In our case just ensured that the values are correct on wincred and am not been able to push it across to the repo.

[jldec]

thank you @ekasey, i was able to workaround the problem using docker-credential-wincred as suggested above and some guidance from https://github.com/kyma-incubator/examples/tree/master/personservice#docker-credential-helper-setup

docker.json (password elided)

{
    "ServerURL": "https://index.docker.io",
    "Username": "jldec",
    "Secret": "****"
}

wincred store

cat docker.json | docker-credential-wincred store
docker-credential-wincred list
{"https://index.docker.io":"jldec","https://index.docker.io/v1/":"jldec"}

build output

default/square-55hjb-pod-670a6f[build-step-export]: 2019/04/03 14:37:38 index.docker.io/jldec/square:latest: digest: sha256:67dbd29303c00536ae5094541429b188765b516b68c36ba2cdd85727763f0658 size: 2857
default/square-55hjb-pod-670a6f[build-step-export]: 2019/04/03 14:37:38
default/square-55hjb-pod-670a6f[build-step-export]: *** Image: docker.io/jldec/square@sha256:67dbd29303c00536ae5094541429b188765b516b68c36ba2cdd85727763f0658

cc: @trisberg

[ekcasey]

We tried to make docker-credential-wincred behave like docker-credential-osxchain (i.e. better) docker/docker-credential-helpers#139. We'll see if they take it.

[ekcasey]

@naavo @jldec @trisberg Our fix for docker-credential-wincred was merged and looks like it will be shipped in the next docker-credential-helpers release.

Discalimer: (I previous lied in this comment and said it was already released b/c I misread the release datetimes)

[ekcasey]

The latest release of docker-credential-helpers includes our fix. Upgrading to the latest docker-credential-wincred should fix this issue. If you still run into trouble after upgrading please feel free to reopen.

[mgibson1121]

Needs @sclevine. Also accepting other pointers for accepting this one that don't involve acquiring a windows machine. Tried using https://www.browserstack.com/, but they're only good for browser and mobile testing of web products.