dockerignore documentation unclear/self-contradictory

[greaber]

I am confused about how dockerignore works. I think it needs a more formal specification, and/or the examples should get into tricky cases more. Here is a slightly pedantic list of some unclear things in the online documentation:

"Filepaths in .dockerignore are absolute with the current directory as the root." Yet in the example .dockerignore file you give, either all or some of the patterns only match relative paths (depending on whether a leading asterisk can match the empty string). I think what is meant is that filepaths are interpreted relative to the current directory but as if a chroot command had made that directory the root?

"the search is not recursive". What does this mean?

"Globbing (file name expansion) is done using Go’s filepath.Match rules." In the linked documentation, we read that asterisk "matches any sequence of non-Separator characters" (which means characters other than '/'?) and that "Match requires pattern to match all of name, not just a substring". Yet in the example file, you say that the pattern "*.md" will exclude all markdown files. All four of these claims can't be true.

"Formats like [^temp*] are ignored." I don't understand what this means. Is it a qualification of the claim that globbing is done using Go's filepath.Match rules?

[cpuguy83]

Would you be able to make a PR to adjust the documents?
I think the issue here is that the dockerignore functionality has expanded a few times w/o a full doc review.

[lukaso]

Wholeheartedly agree with @greaber. I can't get dockerignore to exclude full directories. And the go documentation around filepath.Match is also not useful in helping me.

[thaJeztah]

I think there's actually a PR open that adds a bit more info, will look it up and add a link later

[lukaso]

This (miraculously, but horribly) appears to work!

vendor/bundle/*/*/*/*/*/*/*/*
vendor/bundle/*/*/*/*/*/*/*
vendor/bundle/*/*/*/*/*/*
vendor/bundle/*/*/*/*/*
vendor/bundle/*/*/*/*
vendor/bundle/*/*/*
vendor/bundle/*/*
vendor/bundle/*
vendor/bundle
.git/*/*/*/*/*/*/*/*
.git/*/*/*/*/*/*/*
.git/*/*/*/*/*/*
.git/*/*/*/*/*
.git/*/*/*/*
.git/*/*/*
.git/*/*
.git/*
.git
.vagrant/*/*/*/*/*/*/*/*
.vagrant/*/*/*/*/*/*/*
.vagrant/*/*/*/*/*/*
.vagrant/*/*/*/*/*
.vagrant/*/*/*/*
.vagrant/*/*/*
.vagrant/*/*
.vagrant/*
.vagrant
.sonar/*/*/*/*/*/*/*/*
.sonar/*/*/*/*/*/*/*
.sonar/*/*/*/*/*/*
.sonar/*/*/*/*/*
.sonar/*/*/*/*
.sonar/*/*/*
.sonar/*/*
.sonar/*
.sonar
.ebextensions/*/*/*/*/*/*/*/*
.ebextensions/*/*/*/*/*/*/*
.ebextensions/*/*/*/*/*/*
.ebextensions/*/*/*/*/*
.ebextensions/*/*/*/*
.ebextensions/*/*/*
.ebextensions/*/*
.ebextensions/*
.ebextensions
.bundle/*/*/*/*/*/*/*/*
.bundle/*/*/*/*/*/*/*
.bundle/*/*/*/*/*/*
.bundle/*/*/*/*/*
.bundle/*/*/*/*
.bundle/*/*/*
.bundle/*/*
.bundle/*
.bundle
log/*/*/*/*/*/*/*/*
log/*/*/*/*/*/*/*
log/*/*/*/*/*/*
log/*/*/*/*/*
log/*/*/*/*
log/*/*/*
log/*/*
log/*
log
tmp/*/*/*/*/*/*/*/*
tmp/*/*/*/*/*/*/*
tmp/*/*/*/*/*/*
tmp/*/*/*/*/*
tmp/*/*/*/*
tmp/*/*/*
tmp/*/*
tmp/*
tmp

[thaJeztah]

Found the PR; #14095

[charleswhchan]

PR #14095 appeared to be merged.

@greaber, @lukaso can you review the latest docs is good or if there are still areas that needs improvements?

[greaber]

#14095 helps a lot. There are still some areas that I think could stand improvement. I took a shot at fixing it here. If you understand exactly how .dockerignore works, does my description look correct to you?

[charleswhchan]

Disclaimer: I am no expert on .dockerignore, so would be good to have extra pairs of eyes to review.

@greaber: I had a look at your edits and it is clear to me. I think a slight improvement could be made by splitting up the wildcard and the exclusion sections. ie. Start off with the basics first:

... section for filePath.Match
... example
... table explanation

... section for exclusion rules
... example
... table explanation

Thoughts?

[thaJeztah]

@greaber could you create a pull request with your changes; it's easier to review then, and we can discuss the changes on the PR. Thanks in advance!

[greaber]

#15265. @charleswhchan, I didn't add a second table because I didn't want to make the section any longer than strictly necessary, but the section does discuss the basic case before the negated pattern case.

[charleswhchan]

@greaber: No worries. Simply a suggestion.

[lukaso]

@greater I still think the most common case is to exclude a directory and all its subdirectories. It would be great to include my hacky workaround in the docs since it's not obvious how to do it.

[lukaso]

@greaber I still think the most common case is to exclude a directory and all its subdirectories. It would be great to include my hacky workaround in the docs since it's not obvious how to do it.

[greaber]

@lukaso, I'm confused about your example. Either I am not understanding your issue or I am getting different behavior. Is the following session consistent with what you are seeing?

vagrant@vagrant-ubuntu-vivid-64:~$ cat test.sh
#!/bin/bash

rm -rf foo
mkdir -p foo/a/b/c
mkdir -p foo/b/c/d/e

cat >foo/Dockerfile <<EOF
FROM ubuntu
ADD . /foo
WORKDIR foo
EOF

cat >foo/.dockerignore <<EOF
a
b/c
EOF

docker build -t foo foo
docker run foo find .

vagrant@vagrant-ubuntu-vivid-64:~$ ./test.sh
Sending build context to Docker daemon 3.584 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
 ---> d2a0ecffe6fa
Step 1 : ADD . /foo
 ---> 79ec1503de50
Removing intermediate container a7ff9e7efb7a
Step 2 : WORKDIR foo
 ---> Running in 8200d71cc8de
 ---> 5818058361b1
Removing intermediate container 8200d71cc8de
Successfully built 5818058361b1
.
./.dockerignore
./b
./Dockerfile
vagrant@vagrant-ubuntu-vivid-64:~$

[greaber]

vagrant@vagrant-ubuntu-vivid-64:~$ docker --version
Docker version 1.7.1, build 786b29d

[lukaso]

That looks like it's working properly. In my case the amount of data being transferred was massive, so it might be that the ignore is being done on the docker daemon side? I didn't do the find test.

[greaber]

The ignore is done on the client side except for the special cases of ignoring Dockerfile and .dockerignore.

[lukaso]

I just tested my hypothesis, but appears to work correctly. Here's my slightly modified test:

#!/bin/bash

rm -rf foo
mkdir -p foo/a/b/c
mkdir -p foo/b/c/d/e
openssl rand -out foo/a/b/c/file.txt -base64 $(( 2**30 * 3/4 ))

cat >foo/Dockerfile <<EOF
FROM ubuntu
ADD . /foo
WORKDIR foo
EOF

cat >foo/.dockerignore <<EOF
a
b/c
EOF

docker build -t foo foo
docker run foo find .

$ ./test.sh
Sending build context to Docker daemon 3.584 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
 ---> 63e3c10217b8
Step 1 : ADD . /foo
 ---> 81df29f2035a
Removing intermediate container f101dcf205b8
Step 2 : WORKDIR foo
 ---> Running in dff5c130dd22
 ---> df02d72310ac
Removing intermediate container dff5c130dd22
Successfully built df02d72310ac
.
./Dockerfile
./.dockerignore
./b

The file creation was slow, but it clearly wasn't pushing the 1GB file over to boot2docker. Thanks!

[greaber]

I'm glad things are working for you now! I'm closing the issue now, but let me know if you have any further thoughts on the docs or see any other anomalies.

[lukaso]

@greaber I just realized my problem is with docker-compose. Here is the issue I logged: docker/compose#2098