New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker history reports missing container ids after upgrade to 1.10 #20131

Closed
horkhe opened this Issue Feb 8, 2016 · 7 comments

Comments

Projects
None yet
4 participants
@horkhe

horkhe commented Feb 8, 2016

After upgrade to docker 1.10.0 output of the docker history command started to look like this:

sudo docker history -q registry.postgun.com:5000/mailgun/kafka:0.8.2.2
f2ffd5710981
<missing>
<missing>
<missing>
<missing>
<missing>

As you can see only the top image id is reported properly. Note that this issue is not image specific, output like that we get for all our images.

EDIT:

This result has been observed on both VirtualBox and Rackspace Cloud Server.

The following output was obtained from a Rackspace Cloud Server:

$ docker version
Client:
 Version:      1.10.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   590d5108
 Built:        Thu Feb  4 18:32:05 2016
 OS/Arch:      linux/amd64
Server:
 Version:      1.10.0
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   590d5108
 Built:        Thu Feb  4 18:32:05 2016
 OS/Arch:      linux/amd64
$ docker info
Containers: 4
 Running: 4
 Paused: 0
 Stopped: 0
Images: 31
Server Version: 1.10.0
Storage Driver: aufs
 Root Dir: /var/mailgun/docker/aufs
 Backing Filesystem: extfs
 Dirs: 60
 Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: host bridge null
Kernel Version: 3.13.0-77-generic
Operating System: Ubuntu precise (12.04.4 LTS)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 29.45 GiB
Name: ci.dfw.definbox.com
ID: 45W7:IA4T:XPDT:G4SV:U6XA:KTWY:YVVZ:SD4E:FDXT:SSXY:NAIZ:I46U
WARNING: No swap limit support
@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 10, 2016

Member

I think this is expected; the content-addressable storage no longer uses "parent" images to chain the image-layers together. Newly pulled images also no longer will show the intermediate images (these "missing" images will only be shown for images that were present on the host, but have been migrated to the new storage)

/cc @tonistiigi just for checking; but I think this is expected.

(perhaps we need to mention this in the migration wiki https://github.com/docker/docker/wiki/Engine-v1.10.0-content-addressability-migration)

Member

thaJeztah commented Feb 10, 2016

I think this is expected; the content-addressable storage no longer uses "parent" images to chain the image-layers together. Newly pulled images also no longer will show the intermediate images (these "missing" images will only be shown for images that were present on the host, but have been migrated to the new storage)

/cc @tonistiigi just for checking; but I think this is expected.

(perhaps we need to mention this in the migration wiki https://github.com/docker/docker/wiki/Engine-v1.10.0-content-addressability-migration)

@tonistiigi

This comment has been minimized.

Show comment
Hide comment
@tonistiigi

tonistiigi Feb 10, 2016

Member

@thaJeztah Yes, you are correct. We don't pull parent images anymore. You only have them if you built the image yourself or if you migrated your old image chains.

Maybe we should replace missing with none so it looks less like an error.

Member

tonistiigi commented Feb 10, 2016

@thaJeztah Yes, you are correct. We don't pull parent images anymore. You only have them if you built the image yourself or if you migrated your old image chains.

Maybe we should replace missing with none so it looks less like an error.

@horkhe

This comment has been minimized.

Show comment
Hide comment
@horkhe

horkhe Feb 10, 2016

Hmm, but those layers still have ids, right (inferred from content or not), so why not show them?

Here is the problem: imagine we have myimage:latest that is build of ubuntu:14.04. We want to rebuild myimage:latest only if ubuntu:14.04 changes, so before building we pull ubuntu:14.04 and check if its Id is present in the history of myimage:latest. If it is not, then myimage:latest is rebuilt. Given that now history command of now help, is there a way to check that one image is based of another?

horkhe commented Feb 10, 2016

Hmm, but those layers still have ids, right (inferred from content or not), so why not show them?

Here is the problem: imagine we have myimage:latest that is build of ubuntu:14.04. We want to rebuild myimage:latest only if ubuntu:14.04 changes, so before building we pull ubuntu:14.04 and check if its Id is present in the history of myimage:latest. If it is not, then myimage:latest is rebuilt. Given that now history command of now help, is there a way to check that one image is based of another?

@tonistiigi

This comment has been minimized.

Show comment
Hide comment
@tonistiigi

tonistiigi Feb 10, 2016

Member

@horkhe There is no way to do this securely(the previous method wasn't secure either). We can expose the layer IDs in docker inspect (probably will happen in v1.11) but even that would just confirm the origin of the filesystem data and not image configuration. I'd recommend you to include a comment in your build with a parent image ID or manifest digest. Then you can use check when this id changes from registry/notary and rerun your build if needed.

Member

tonistiigi commented Feb 10, 2016

@horkhe There is no way to do this securely(the previous method wasn't secure either). We can expose the layer IDs in docker inspect (probably will happen in v1.11) but even that would just confirm the origin of the filesystem data and not image configuration. I'd recommend you to include a comment in your build with a parent image ID or manifest digest. Then you can use check when this id changes from registry/notary and rerun your build if needed.

@marius311

This comment has been minimized.

Show comment
Hide comment
@marius311

marius311 Mar 13, 2016

Sorry, I don't totally know how the internals work, but does there currently exist a workaround? By that I mean, can I pull an image, then run it at a layer other than the top layer? (I basically use this for testing purposes, its certainly possible to build the image myself then do it, but its slightly less convenient)

marius311 commented Mar 13, 2016

Sorry, I don't totally know how the internals work, but does there currently exist a workaround? By that I mean, can I pull an image, then run it at a layer other than the top layer? (I basically use this for testing purposes, its certainly possible to build the image myself then do it, but its slightly less convenient)

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Mar 13, 2016

Member

@marius311 not at this moment, follow the discussion on #20316, which talks about this some more

Member

thaJeztah commented Mar 13, 2016

@marius311 not at this moment, follow the discussion on #20316, which talks about this some more

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Mar 13, 2016

Member

I'll close this issue in favor of #20316, so that we don't have too many similar issues open in this subject

Member

thaJeztah commented Mar 13, 2016

I'll close this issue in favor of #20316, so that we don't have too many similar issues open in this subject

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment