New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reset properties inherited from parent image #3465

Open
shykes opened this Issue Jan 6, 2014 · 117 comments

Comments

Projects
None yet
@shykes
Collaborator

shykes commented Jan 6, 2014

When building an image I may want to reset some of its properties instead of inheriting them from the parent image. It makes sense to inherit all properties by default, but there should be a way to explicitly and selectively reset them when it makes sense.

This is a more generic solution to #2210 which only addresses expose.

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Jan 6, 2014

Collaborator

Suggestions welcome for syntax.

Collaborator

shykes commented Jan 6, 2014

Suggestions welcome for syntax.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jan 6, 2014

Member

The best I can come up with is corresponding commands like UNVOLUME or more generically -VOLUME (but that would add more confusion, and potentially even create the misconception that +VOLUME should work, and should work differently from just VOLUME).

I very definitely want such a thing (most especially for VOLUMEs). It's also a little disorienting that things like VOLUME apply to following RUN lines, but things like ENTRYPOINT don't. Sometimes that's very useful, and sometimes it's not, but a generic "disable previous X instruction" could solve the issues around that quite nicely.

Member

tianon commented Jan 6, 2014

The best I can come up with is corresponding commands like UNVOLUME or more generically -VOLUME (but that would add more confusion, and potentially even create the misconception that +VOLUME should work, and should work differently from just VOLUME).

I very definitely want such a thing (most especially for VOLUMEs). It's also a little disorienting that things like VOLUME apply to following RUN lines, but things like ENTRYPOINT don't. Sometimes that's very useful, and sometimes it's not, but a generic "disable previous X instruction" could solve the issues around that quite nicely.

@dergachev

This comment has been minimized.

Show comment
Hide comment
@dergachev

dergachev Mar 26, 2014

Is there a workaround for this in the meantime? I'm extending an image with an ENTRYPOINT (https://github.com/jagregory/pandoc-docker/blob/master/Dockerfile) and I need to unset the entrypoint. I tried using the following in my Dockerfile:

FROM jagregory/pandoc
ENTRYPOINT [] # this basically gets ignored (bug?)

FROM jagregory/pandoc
ENTRYPOINT [""] # this will make docker try to exec '' (the empty string)

FROM jagregory/pandoc
ENTRYPOINT ["/bin/sh", "-c"] 
# this will only work if docker run args are quoted:
#   docker run dergachev/pandoc "echo a b c"

Thanks!

dergachev commented Mar 26, 2014

Is there a workaround for this in the meantime? I'm extending an image with an ENTRYPOINT (https://github.com/jagregory/pandoc-docker/blob/master/Dockerfile) and I need to unset the entrypoint. I tried using the following in my Dockerfile:

FROM jagregory/pandoc
ENTRYPOINT [] # this basically gets ignored (bug?)

FROM jagregory/pandoc
ENTRYPOINT [""] # this will make docker try to exec '' (the empty string)

FROM jagregory/pandoc
ENTRYPOINT ["/bin/sh", "-c"] 
# this will only work if docker run args are quoted:
#   docker run dergachev/pandoc "echo a b c"

Thanks!

@stormbeta

This comment has been minimized.

Show comment
Hide comment
@stormbeta

stormbeta Jun 9, 2014

I would definitely like to have some way of removing VOLUME points inherited from parent images.

For example, suppose I had a main image for an application that used an external mountpoint for persistent data, but I also wanted an image based on it that was pre-populated with test data instead. As-is, I can't do that if the parent image used VOLUME because any changes/additions to those directories, even if those changes are during a docker build, are lost on commit.

stormbeta commented Jun 9, 2014

I would definitely like to have some way of removing VOLUME points inherited from parent images.

For example, suppose I had a main image for an application that used an external mountpoint for persistent data, but I also wanted an image based on it that was pre-populated with test data instead. As-is, I can't do that if the parent image used VOLUME because any changes/additions to those directories, even if those changes are during a docker build, are lost on commit.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jun 30, 2014

Member

Just to update from @dergachev's comment, CMD [] and ENTRYPOINT [] were working last time I tested them recently, and should still be working (anything else would be ripe for bug filing).

Member

tianon commented Jun 30, 2014

Just to update from @dergachev's comment, CMD [] and ENTRYPOINT [] were working last time I tested them recently, and should still be working (anything else would be ripe for bug filing).

@mmerickel

This comment has been minimized.

Show comment
Hide comment
@mmerickel

mmerickel Jun 30, 2014

You can reset all of the single-option commands via

ENTRYPOINT []
CMD []
USER 0
WORKDIR /

This should leave the remaining, un-resettable metadata as ENV, VOLUME, EXPOSE and maybe ONBUILD.

mmerickel commented Jun 30, 2014

You can reset all of the single-option commands via

ENTRYPOINT []
CMD []
USER 0
WORKDIR /

This should leave the remaining, un-resettable metadata as ENV, VOLUME, EXPOSE and maybe ONBUILD.

@codeon-nat

This comment has been minimized.

Show comment
Hide comment
@codeon-nat

codeon-nat Jan 10, 2015

(This is coming from #8709)

If I exposed sockets 9000-9002 in the parent, but needed to unexpose 9001 in the child, I'd then have to write in the style of "unsetting"

EXPOSE
EXPOSE 9000
EXPOSE 9002

which would work but

UNEXPOSE 9001

looks nicer.

An advantage being, that it doesn't affect any EXPOSEs from further up the inheritance chain, which I might want to add later.

codeon-nat commented Jan 10, 2015

(This is coming from #8709)

If I exposed sockets 9000-9002 in the parent, but needed to unexpose 9001 in the child, I'd then have to write in the style of "unsetting"

EXPOSE
EXPOSE 9000
EXPOSE 9002

which would work but

UNEXPOSE 9001

looks nicer.

An advantage being, that it doesn't affect any EXPOSEs from further up the inheritance chain, which I might want to add later.

@AshleyAitken

This comment has been minimized.

Show comment
Hide comment
@AshleyAitken

AshleyAitken commented Jan 11, 2015

@icecrime

This comment has been minimized.

Show comment
Hide comment
@icecrime

icecrime Apr 15, 2015

Contributor

This has been discussed in #8177, we're closing this for a lack of real world use cases.

Contributor

icecrime commented Apr 15, 2015

This has been discussed in #8177, we're closing this for a lack of real world use cases.

@icecrime icecrime closed this Apr 15, 2015

@mitar

This comment has been minimized.

Show comment
Hide comment
@mitar

mitar Apr 16, 2015

Why is this being closed? There were 9 people commenting here. I think this would be a really useful thing to have. The real world use cases are being able to build upon existing images with ease. Sometimes you want to add property, sometimes you want to remove it. This is normal.

mitar commented Apr 16, 2015

Why is this being closed? There were 9 people commenting here. I think this would be a really useful thing to have. The real world use cases are being able to build upon existing images with ease. Sometimes you want to add property, sometimes you want to remove it. This is normal.

@kamermans

This comment has been minimized.

Show comment
Hide comment
@kamermans

kamermans May 18, 2015

I agree, for example, I'm extending the nginx image for an SSL offloader and I want to UNEXPOSE 80 but leave 443.

kamermans commented May 18, 2015

I agree, for example, I'm extending the nginx image for an SSL offloader and I want to UNEXPOSE 80 but leave 443.

@seth-admittedly

This comment has been minimized.

Show comment
Hide comment
@seth-admittedly

seth-admittedly Jun 4, 2015

Being able to unexpose ports is pretty critical if you want to run multiple instances of, e.g., nginx. Without this feature, each image that inherits from nginx tries to expose 80 and 443, causing the error Bind for 0.0.0.0:80 failed: port is already allocated.

Nevermind, this was just poor configuration on my side.

seth-admittedly commented Jun 4, 2015

Being able to unexpose ports is pretty critical if you want to run multiple instances of, e.g., nginx. Without this feature, each image that inherits from nginx tries to expose 80 and 443, causing the error Bind for 0.0.0.0:80 failed: port is already allocated.

Nevermind, this was just poor configuration on my side.

@apatrida

This comment has been minimized.

Show comment
Hide comment
@apatrida

apatrida Jul 22, 2015

(April 15: "No real world use cases" I'm surprised you cannot imagine at least one and closed this)

I have a base image that exposes volumes or ports for optional software, then FROM that in another Dockerfile to make an image that should not expose anything it doesn't want to, or even things that it has uninstalled from the ancestor. Why would we NOT want to be able to remove these settings?

apatrida commented Jul 22, 2015

(April 15: "No real world use cases" I'm surprised you cannot imagine at least one and closed this)

I have a base image that exposes volumes or ports for optional software, then FROM that in another Dockerfile to make an image that should not expose anything it doesn't want to, or even things that it has uninstalled from the ancestor. Why would we NOT want to be able to remove these settings?

@download13

This comment has been minimized.

Show comment
Hide comment
@download13

download13 Aug 10, 2015

I also have a use case for this. I want to be able to create an image containing a database snapshot, but all the mysql packages have VOLUME /var/lib/mysql set. It would nice to be able to turn off the volume the changes to the database made in my Dockerfile will stick with the image.

The only other option is to completely re-create a custom mysql image, but that somehow seems wasteful since plenty of other people have already put together better default mysql servers than I could.

download13 commented Aug 10, 2015

I also have a use case for this. I want to be able to create an image containing a database snapshot, but all the mysql packages have VOLUME /var/lib/mysql set. It would nice to be able to turn off the volume the changes to the database made in my Dockerfile will stick with the image.

The only other option is to completely re-create a custom mysql image, but that somehow seems wasteful since plenty of other people have already put together better default mysql servers than I could.

@tomheinan

This comment has been minimized.

Show comment
Hide comment
@tomheinan

tomheinan Aug 10, 2015

Adding an additional use case - I'm inheriting from the official RabbitMQ image, but I only want to expose websocket ports (80 and 443) and not the default AMQP port (5672). Seems like this should be a pretty reasonable thing to want to do?

tomheinan commented Aug 10, 2015

Adding an additional use case - I'm inheriting from the official RabbitMQ image, but I only want to expose websocket ports (80 and 443) and not the default AMQP port (5672). Seems like this should be a pretty reasonable thing to want to do?

@thaJeztah thaJeztah referenced this issue Sep 24, 2015

Merged

Add volume api #14242

3 of 3 tasks complete
@praqma-thi

This comment has been minimized.

Show comment
Hide comment
@praqma-thi

praqma-thi Nov 20, 2015

Adding another use case. I want to build a git test environment using the gogs image but it's tedious to have the data persist since it's all stored in a volume. It'd be great if I'd be able to simply UNVOLUME the volume and build my image after setting up the environment.

praqma-thi commented Nov 20, 2015

Adding another use case. I want to build a git test environment using the gogs image but it's tedious to have the data persist since it's all stored in a volume. It'd be great if I'd be able to simply UNVOLUME the volume and build my image after setting up the environment.

@krasi-georgiev

This comment has been minimized.

Show comment
Hide comment
@krasi-georgiev

krasi-georgiev Nov 20, 2015

Contributor

+1

inheriting from the official php and want to use sockets instead of ports so need to remove the exposed 9000 port

Contributor

krasi-georgiev commented Nov 20, 2015

+1

inheriting from the official php and want to use sockets instead of ports so need to remove the exposed 9000 port

@macropin

This comment has been minimized.

Show comment
Hide comment
@macropin

macropin Dec 9, 2015

Anyone who has used Docker in a non-trivial capacity will have found these limitations with inherited containers.

@shykes @icecrime how is this now closed? Is it too hard to solve with the current syntax and need for backwards compatibility? Whats the plan?

macropin commented Dec 9, 2015

Anyone who has used Docker in a non-trivial capacity will have found these limitations with inherited containers.

@shykes @icecrime how is this now closed? Is it too hard to solve with the current syntax and need for backwards compatibility? Whats the plan?

@ianmiell

This comment has been minimized.

Show comment
Hide comment
@ianmiell

ianmiell Dec 22, 2015

+1 - real world use case for EXPOSE override here.

ianmiell commented Dec 22, 2015

+1 - real world use case for EXPOSE override here.

@matthudsonatx

This comment has been minimized.

Show comment
Hide comment
@matthudsonatx

matthudsonatx Feb 12, 2018

My use case comes from docker-compose.yaml: I'd like to have a compose file for development with overrides for production that add a TLS reverse-proxy, Maven repository, takes over PORT 80/443, unexposes port 80 and 5432 that the development compose file exposes. Or a compose file for production with development override.

The add-only nature of layering compose files inherited from Dockerfiles complicates system design. It would be really cool if some parameters could be retracted, or if I simply got a different container built with overrides active. I am not picky how it works under the hood wrt docker-compose.

matthudsonatx commented Feb 12, 2018

My use case comes from docker-compose.yaml: I'd like to have a compose file for development with overrides for production that add a TLS reverse-proxy, Maven repository, takes over PORT 80/443, unexposes port 80 and 5432 that the development compose file exposes. Or a compose file for production with development override.

The add-only nature of layering compose files inherited from Dockerfiles complicates system design. It would be really cool if some parameters could be retracted, or if I simply got a different container built with overrides active. I am not picky how it works under the hood wrt docker-compose.

@juliangamble

This comment has been minimized.

Show comment
Hide comment
@juliangamble

juliangamble Feb 14, 2018

Thanks @thaJeztah - could you please point us to the line of code that in your view is the place to start reading to look at fixing this with a pull request?

juliangamble commented Feb 14, 2018

Thanks @thaJeztah - could you please point us to the line of code that in your view is the place to start reading to look at fixing this with a pull request?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 14, 2018

Member

I haven't worked much on the builder code myself, but changes should likely be in the https://github.com/moby/moby/tree/master/builder package.

Member

thaJeztah commented Feb 14, 2018

I haven't worked much on the builder code myself, but changes should likely be in the https://github.com/moby/moby/tree/master/builder package.

@caruccio

This comment has been minimized.

Show comment
Hide comment
@caruccio

caruccio Feb 23, 2018

This issue has 3 years! Is it so hard to agree on a so basic feature, or am I missing something?

caruccio commented Feb 23, 2018

This issue has 3 years! Is it so hard to agree on a so basic feature, or am I missing something?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Feb 23, 2018

Member

@caruccio yes, you're missing something: scroll up 4 comments #3465 (comment)

Member

thaJeztah commented Feb 23, 2018

@caruccio yes, you're missing something: scroll up 4 comments #3465 (comment)

@neechbear

This comment has been minimized.

Show comment
Hide comment
@neechbear

neechbear Feb 25, 2018

I also have a couple of use cases (one a personal project, and the second a work project) where I wish to overload a VOLUME, EXPOSE and ENTRYPOINT statement from a parent image.

While I have a workaround for ENTRYPOINT by just setting a new empty entry point with ENTRYPOINT [], and can probably learn to live with ignoring EXPOSE, ... I am left scratching my head about how to not inherit VOLUME definitions.

neechbear commented Feb 25, 2018

I also have a couple of use cases (one a personal project, and the second a work project) where I wish to overload a VOLUME, EXPOSE and ENTRYPOINT statement from a parent image.

While I have a workaround for ENTRYPOINT by just setting a new empty entry point with ENTRYPOINT [], and can probably learn to live with ignoring EXPOSE, ... I am left scratching my head about how to not inherit VOLUME definitions.

@themanifold

This comment has been minimized.

Show comment
Hide comment
@themanifold

themanifold Mar 7, 2018

I just came across this issue in my code base where the parent image had a VOLUME, meaning all my changes to this volume in a child image were thrown away. I thought I was going mad for 2 days until I finally found my way to this issue. Please could someone implement this.

themanifold commented Mar 7, 2018

I just came across this issue in my code base where the parent image had a VOLUME, meaning all my changes to this volume in a child image were thrown away. I thought I was going mad for 2 days until I finally found my way to this issue. Please could someone implement this.

@gdraheim

This comment has been minimized.

Show comment
Hide comment
@gdraheim

gdraheim Apr 22, 2018

There is a WORKAROUND.

You can always docker save image -o image.tar, unpack that archive, edit the metadata and repack for a docker load -i image2.tar. That way one can make an image2 that does not have any of the prior VOLUME declarations in it.

Since I have to do those steps quite regularly, I have created a little script to help with task of cleaning up a third party image. Have a look at docker-copyedit

gdraheim commented Apr 22, 2018

There is a WORKAROUND.

You can always docker save image -o image.tar, unpack that archive, edit the metadata and repack for a docker load -i image2.tar. That way one can make an image2 that does not have any of the prior VOLUME declarations in it.

Since I have to do those steps quite regularly, I have created a little script to help with task of cleaning up a third party image. Have a look at docker-copyedit

@joshrivers

This comment has been minimized.

Show comment
Hide comment
@joshrivers

joshrivers Apr 22, 2018

Fantastic work @gdraheim ! A workable solution in <250 lines of python.

joshrivers commented Apr 22, 2018

Fantastic work @gdraheim ! A workable solution in <250 lines of python.

@q0rban

This comment has been minimized.

Show comment
Hide comment
@q0rban

q0rban Apr 23, 2018

@gdraheim wow, this is great! From the README:

The wish to REMOVE ALL VOLUMES came from the fact that I did want to download a tested image for local tests where the data part should be committed to the history as well in order to turn back both program and data to a defined state so that another test run will start off the exact same checkpoint.

This is our use case as well.

q0rban commented Apr 23, 2018

@gdraheim wow, this is great! From the README:

The wish to REMOVE ALL VOLUMES came from the fact that I did want to download a tested image for local tests where the data part should be committed to the history as well in order to turn back both program and data to a defined state so that another test run will start off the exact same checkpoint.

This is our use case as well.

@gdraheim

This comment has been minimized.

Show comment
Hide comment
@gdraheim

gdraheim Apr 29, 2018

I have expanded docker-copyedit to cover all metadata entries of an image, so it can work on all inheritered properties even beyond the problematic cases of the EXPOSE and VOLUME lists. That would be user, workingdir, labels, env settings for things seen often. Copying ENTRYPOINT to CMD is also a modification that I do quite regulary. No need to make an intermediate docker-build step anymore, just go for docker-copyedit. ;)

gdraheim commented Apr 29, 2018

I have expanded docker-copyedit to cover all metadata entries of an image, so it can work on all inheritered properties even beyond the problematic cases of the EXPOSE and VOLUME lists. That would be user, workingdir, labels, env settings for things seen often. Copying ENTRYPOINT to CMD is also a modification that I do quite regulary. No need to make an intermediate docker-build step anymore, just go for docker-copyedit. ;)

@namedgraph

This comment has been minimized.

Show comment
Hide comment
@namedgraph

namedgraph Jun 11, 2018

The time used by the Docker team to track and repeatedly ignore this issue would probably had been enough to fix it instead.

namedgraph commented Jun 11, 2018

The time used by the Docker team to track and repeatedly ignore this issue would probably had been enough to fix it instead.

@boscowitch

This comment has been minimized.

Show comment
Hide comment
@boscowitch

boscowitch Jun 21, 2018

Can we now please reopen this after obviously a ton of users requesting this...
or at least give a reasonable argument against it not just ignoring all the uses cases (I wanna unexpose a port too and only to get a cleaner docker ps output without fucking (80/80/tcp) bevore i will build my own image.... (which is harder for non opensource Dockerfiles)

boscowitch commented Jun 21, 2018

Can we now please reopen this after obviously a ton of users requesting this...
or at least give a reasonable argument against it not just ignoring all the uses cases (I wanna unexpose a port too and only to get a cleaner docker ps output without fucking (80/80/tcp) bevore i will build my own image.... (which is harder for non opensource Dockerfiles)

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jun 21, 2018

Member

The issue is still open; It's open source; contributions are welcome #3465 (comment)

Member

thaJeztah commented Jun 21, 2018

The issue is still open; It's open source; contributions are welcome #3465 (comment)

@nevergone

This comment has been minimized.

Show comment
Hide comment

nevergone commented Jun 21, 2018

@AnthonyMastrean

This comment has been minimized.

Show comment
Hide comment
@AnthonyMastrean

AnthonyMastrean Jun 22, 2018

Am I right thinking that this would be fixed in moby/buildkit now? I see most of the Dockerfile command infrastructure over there.

AnthonyMastrean commented Jun 22, 2018

Am I right thinking that this would be fixed in moby/buildkit now? I see most of the Dockerfile command infrastructure over there.

@AnthonyMastrean

This comment has been minimized.

Show comment
Hide comment
@AnthonyMastrean

AnthonyMastrean Jun 22, 2018

I'm also a fan of UNSET, like

UNSET EXPOSE 9000

or

UNSET LABEL foo

So, I'm looking at commands that have subcommand-ish forms, like the HEALTHCHECK CMD form and I notice that HEALTHCHECK already has an unset form...

HEALTHCHECK NONE

This is an interesting choice, but HEALTHCHECK also only defines 1 configuration (and overrides w/ the newest), it doesn't allow defining multiple, like LABEL, EXPOSE, and VOLUME do.

I just wonder how these should interact or if there's some other kind of NONE form that might work.

AnthonyMastrean commented Jun 22, 2018

I'm also a fan of UNSET, like

UNSET EXPOSE 9000

or

UNSET LABEL foo

So, I'm looking at commands that have subcommand-ish forms, like the HEALTHCHECK CMD form and I notice that HEALTHCHECK already has an unset form...

HEALTHCHECK NONE

This is an interesting choice, but HEALTHCHECK also only defines 1 configuration (and overrides w/ the newest), it doesn't allow defining multiple, like LABEL, EXPOSE, and VOLUME do.

I just wonder how these should interact or if there's some other kind of NONE form that might work.

@smiller171

This comment has been minimized.

Show comment
Hide comment
@smiller171

smiller171 Jun 25, 2018

Some way to remove exposed ports is really needed for controlling what is exposed when using host networking.

smiller171 commented Jun 25, 2018

Some way to remove exposed ports is really needed for controlling what is exposed when using host networking.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment