Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

1.10: problems pulling images from gcr.io #18832

Closed
ibuildthecloud opened this issue Dec 21, 2015 · 4 comments
Closed

1.10: problems pulling images from gcr.io #18832

ibuildthecloud opened this issue Dec 21, 2015 · 4 comments
Labels
area/distribution kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed.
Milestone
1.10.0

Comments

@ibuildthecloud
Copy link
Contributor

With 1.10 I get the following error:

$ docker pull gcr.io/google_containers/etcd:2.0.12
Error response from daemon: manifest unknown: Invalid manifest reference "fafe4735269937ef7ba1fa0badb66d7fd70736619862e3982144c59289fddb11".

Client:
 Version:      1.10.0-dev
 API version:  1.22
 Go version:   go1.5.2
 Git commit:   8537501
 Built:        Mon Dec 21 20:38:58 2015
 OS/Arch:      linux/amd64

Server:
 Version:      1.10.0-dev
 API version:  1.22
 Go version:   go1.5.2
 Git commit:   8537501
 Built:        Mon Dec 21 20:38:58 2015
 OS/Arch:      linux/amd64
darren@inotmac:~/src/ros/src/github.com/rancherio/os$ ~/src/docker/bundles/latest/binary/docker info
Containers: 7
Images: 912
Server Version: 1.10.0-dev
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.2.5
Operating System: Ubuntu 15.10 (containerized)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.4 GiB
Name: inotmac
ID: HMGD:MCSR:XSNO:I7OW:2H7X:KE3J:XIUK:FEOU:SR4J:PVBZ:B3KW:2C6V
Debug mode (server): true
 File Descriptors: 29
 Goroutines: 49
 System Time: 2015-12-21T13:43:18.425569734-07:00
 EventsListeners: 1
 Init SHA1: 
 Init Path: /home/darren/src/docker/bundles/latest/binary/docker
 Docker Root Dir: /var/lib/docker
Username: ibuildthecloud
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
@thaJeztah thaJeztah added area/distribution kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. labels Dec 21, 2015
@thaJeztah
Copy link
Member

ping @tonistiigi @aaronlehmann

(Thanks for reporting @ibuildthecloud )

@ibuildthecloud
Copy link
Contributor Author

Just in case this helps. More logs

DEBU[0032] Calling POST /v1.22/images/create            
DEBU[0032] POST /v1.22/images/create?fromImage=gcr.io%2Fgoogle_containers%2Fetcd%3A2.0.12 
DEBU[0032] hostDir: /etc/docker/certs.d/gcr.io          
DEBU[0032] hostDir: /etc/docker/certs.d/gcr.io          
DEBU[0032] Trying to pull gcr.io/google_containers/etcd from https://gcr.io v2 
DEBU[0033] Pulling ref from V2 registry: "2.0.12"       
DEBU[0034] v2 error: errcode.Error manifest unknown: Invalid manifest reference "fafe4735269937ef7ba1fa0badb66d7fd70736619862e3982144c59289fddb11". 
DEBU[0034] Error trying v2 registry: manifest unknown: Invalid manifest reference "fafe4735269937ef7ba1fa0badb66d7fd70736619862e3982144c59289fddb11". 
DEBU[0034] Skipping v1 endpoint https://gcr.io because v2 registry was detected 
ERRO[0034] Handler for POST /v1.22/images/create returned error: manifest unknown: Invalid manifest reference "fafe4735269937ef7ba1fa0badb66d7fd70736619862e3982144c59289fddb11". 
Error response from daemon: manifest unknown: Invalid manifest reference "fafe4735269937ef7ba1fa0badb66d7fd70736619862e3982144c59289fddb11".

@tonistiigi tonistiigi added this to the 1.10 milestone Dec 21, 2015
@aaronlehmann
Copy link
Contributor

Could it be that gcr.io is only serving this image using the v1 protocol, and #18590 prevents fallback to the v1 protocol in this case?

cc @dmp42

@ibuildthecloud
Copy link
Contributor Author

@aaronlehmann I think your suspicion is correct. I reverted #18590 locally and now it is working.

@aaronlehmann aaronlehmann mentioned this issue Dec 21, 2015
Merged
aaronlehmann added a commit to aaronlehmann/docker that referenced this issue Dec 22, 2015
@aaronlehmann
When a manifest is not found, allow fallback to v1
9d6acbe 
PR moby#18590 caused compatibility issues with registries such as gcr.io
which support both the v1 and v2 protocols, but do not provide the same
set of images over both protocols. After moby#18590, pulls from these
registries would never use the v1 protocol, because of the
Docker-Distribution-Api-Version header indicating that v2 was supported.

Fix the problem by making an exception for the case where a manifest is
not found. This should allow fallback to v1 in case that image is
exposed over the v1 protocol but not the v2 protocol.

This avoids the overly aggressive fallback behavior before moby#18590 which
would allow protocol fallback after almost any error, but restores
interoperability with mixed v1/v2 registry setups.

Fixes moby#18832

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
aditirajagopal pushed a commit to aditirajagopal/docker that referenced this issue Feb 8, 2016
@aaronlehmann
When a manifest is not found, allow fallback to v1
e719669 
PR moby#18590 caused compatibility issues with registries such as gcr.io
which support both the v1 and v2 protocols, but do not provide the same
set of images over both protocols. After moby#18590, pulls from these
registries would never use the v1 protocol, because of the
Docker-Distribution-Api-Version header indicating that v2 was supported.

Fix the problem by making an exception for the case where a manifest is
not found. This should allow fallback to v1 in case that image is
exposed over the v1 protocol but not the v2 protocol.

This avoids the overly aggressive fallback behavior before moby#18590 which
would allow protocol fallback after almost any error, but restores
interoperability with mixed v1/v2 registry setups.

Fixes moby#18832

Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
@joshuacox joshuacox mentioned this issue Sep 6, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/distribution kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed.
Projects
None yet
Milestone
1.10.0
Development

No branches or pull requests
4 participants
@tonistiigi @ibuildthecloud @thaJeztah @aaronlehmann