Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile ADD command does not follow symlinks on host #1676

Closed
fufie opened this issue Aug 26, 2013 · 24 comments
Closed

Dockerfile ADD command does not follow symlinks on host #1676

fufie opened this issue Aug 26, 2013 · 24 comments

Comments

@fufie
Copy link

@fufie fufie commented Aug 26, 2013

Docker version 0.6.1. Tested on ubuntu.

I have a Dockerfile with the following statement:

ADD my-archive.tar.gz /archive/my-archive.tar.gz

where my-archive.tar.gz is a symlink from the build directory on the host system to a valid and readable my-archive.tar.gz archive. The /archive path on the destination/container exists.

Result:
Error build: ... No such file or directory

If my-archive.tar.gz is a hardlink things work just fine.

Symlinking is common practice when joining up resources on the filesystem for a build, and as such should be followed.

@fufie

This comment has been minimized.

Copy link
Author

@fufie fufie commented Aug 26, 2013

Got a thorough discussion and clarification the irc channel that made sense. :-)

@fufie fufie closed this Aug 26, 2013
@sgargan

This comment has been minimized.

Copy link

@sgargan sgargan commented Sep 5, 2013

What was the outcome of the discussion? I'd like to ADD symlink'd files in my Dockerfiles too, is there a reason why this is not advisable?

@crosbymichael

This comment has been minimized.

Copy link
Member

@crosbymichael crosbymichael commented Sep 5, 2013

@sgargan We do not allow this because it's not repeatable. A symlink on your machine is the not the same as my machine and the same Dockerfile would produce two different results. Also having symlinks to /etc/paasswd would cause issues because it would link the host files and not your local files.

@sgargan

This comment has been minimized.

Copy link

@sgargan sgargan commented Sep 5, 2013

I'm definitely not advocating symlinking to a file on the host. Really what I'm talking about is adding the contents of a symlink'd file in the build to a concrete path in the image. I typically have all my docker/devops files in a git repo with relative path symlinks where needed so I'm not repeating myself. If you have a clone of the repo, you have the symlink too, so in this case the symlinks are same on all machines. I'd like to be able to use ADD to pull the contents of a file symlink'd next to the docker file and write the contents into the image not the symlink.

e.g.

common/somecommon

containers/spiffy_container/Dockerfile
containers/spiffy_container/somecommon

where
ls -al somecommon
lrwxrwxrwx 1 vagrant vagrant 20 Sep 5 18:33 somecommon -> ../common/somecommon

Then in the docker file

add somecommon /etc/concrete_somecommon

after which /etc/concrete_somecommon would be a normal file with the contents of the file symlink'd in the build folder.

I would definitely find it useful to enable this even if it came with a warning that its against the general accepted practice.

Does this sound unreasonable? am I missing something?

@realyze

This comment has been minimized.

Copy link

@realyze realyze commented Sep 17, 2013

+1 this. I want my Dockerfile to be in a subdirectory of my project and I'd like to create a symlink to the parent directory so that I can add that to the image (I can't add it directly, see #1280).

@jbenet

This comment has been minimized.

Copy link

@jbenet jbenet commented Jan 14, 2014

+1 to this as well. symlinks (and parent directories) should be ADDable. Lots of project organization structures are precluded by this-- particularly problematic when additional git-clones are non trivial (private repos).

@jordansissel

This comment has been minimized.

Copy link
Contributor

@jordansissel jordansissel commented Jan 21, 2014

+1 on this. My use case is the same as @sgargan and @jbenet. I want to share things among dockerfiles. The most obvious solution to me was to try symlinks, which don't work, and googling dockerfile add symlink presents this ticket as the first hit.

Can this be reopened and reconsidered, please?

@tianon

This comment has been minimized.

Copy link
Member

@tianon tianon commented Jan 21, 2014

The solution to that is going to be Dockerfile dependency. For example, you'll have many subdirectories that each contain Dockerfiles that may or may not depend on each other or on a common directory (like a common library of code). There will then be a Dockerfile at the root (common) level that handles the "orchestration" of building these sub-Dockerfiles correctly with the common context that they need.

@jbenet

This comment has been minimized.

Copy link

@jbenet jbenet commented Jan 21, 2014

@tianon recursive Dockerfiles are cool! But, forced recursive Dockerfiles aren't. Look into the mess that recursive Makefiles can create.

More importantly, this is does not solve the problem-- not even remotely. Projects have very particular directory structures, often imposed by a variety of organizational, cultural, and even system constraints. These structures are blown up dramatically when considering how companies handle their own vm images and provisioning. Things like configuration files, auth keys, etc are distributed independently of the source code itself, and often referenced or symlinked in. These things are not in the same repository, and often not even handled by the same engineers. Docker forcing a particular directory structure, makes it cumbersome to use and a pain to work with. This is not the usual Docker smart systems design we're used to. Symlinks are really just files, for good reason. Docker should treat them so, just like tar, and all sorts of file distribution tools do.

@kenichi

This comment has been minimized.

Copy link

@kenichi kenichi commented Jan 29, 2014

+1 - I understand the not-repeatable concept, but there should at least be a flag to allow it for cases where users Know What They Are Doing.

@orospakr

This comment has been minimized.

Copy link

@orospakr orospakr commented Feb 19, 2014

+1 - Just ran headlong into this while writing capistrano tasks for building containers out of slugs produced elsewhere in my build process.

Dockerfiles already don't guarantee repeatability; any RUN command that uses the network sees to that.

@newhoggy

This comment has been minimized.

Copy link

@newhoggy newhoggy commented Mar 11, 2014

+1

@newhoggy

This comment has been minimized.

Copy link

@newhoggy newhoggy commented Mar 11, 2014

@tianon, is there an example somewhere that demonstrates how to do Dockerfile dependencies?

@tianon

This comment has been minimized.

Copy link
Member

@tianon tianon commented Mar 12, 2014

The feature doesn't exist quite yet; it's still on the drawing board waiting for a solid implementation.

@username99987

This comment has been minimized.

Copy link

@username99987 username99987 commented Mar 12, 2014

+1

@pauldub

This comment has been minimized.

Copy link

@pauldub pauldub commented Mar 19, 2014

Well this is an issue for me too for other reasons, it happens I have a project stored with git-annex so it is symlinks everywhere, I already workaround it with a small script that echoes those back to real files but I would love to be able to pass a flag to the docker build command like --follow-symlinks or whatever for this specific case.

@max-zelinski

This comment has been minimized.

Copy link

@max-zelinski max-zelinski commented May 2, 2014

+1 to @pauldub suggestion
just ran into this limitation. dockerfile is already unrepeatable.

@clsung

This comment has been minimized.

Copy link

@clsung clsung commented May 12, 2014

+1 here

@iflowfor8hours

This comment has been minimized.

Copy link

@iflowfor8hours iflowfor8hours commented May 12, 2014

Having problems with this issue as well. I understand the security implications of not adding parent directories, but I would like to be able to have a common build directory for multiple dockerfiles in a repo.

@buster

This comment has been minimized.

Copy link

@buster buster commented Jun 5, 2014

I have the same problem, it would be very good to have atleast a switch that enables it somewhere..
I have multiple Dockerfiles and a common set of many files which are actually the same for those Dockerfiles.

@marthjod

This comment has been minimized.

Copy link

@marthjod marthjod commented Jun 5, 2014

+1 for ADDable symlinks.

@reberhardt7

This comment has been minimized.

Copy link

@reberhardt7 reberhardt7 commented Jul 7, 2014

+1 for the reason @jbenet gave...

@elgalu

This comment has been minimized.

Copy link
Contributor

@elgalu elgalu commented Jul 20, 2014

+1 for ADDable symlinks.

@glenbot

This comment has been minimized.

Copy link

@glenbot glenbot commented Jul 22, 2014

+1 symlinks make sense to allow for sharable build directories without having to have the same files in each context. Let us decide how to add stuff to our own builds. Don't try and protect.

@moby moby locked and limited conversation to collaborators Jul 22, 2014
luan-cestari added a commit to lightblue-platform/lightblue-dockerfiles that referenced this issue Nov 10, 2014
Error  failed to build: Forbidden path outside the build context . More info moby/moby#1676
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.