Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ignoring some files when building Docker images #1224

Merged
merged 3 commits into from Jan 12, 2021
Merged

Conversation

yvanzo
Copy link
Contributor

@yvanzo yvanzo commented Jan 4, 2021

Problem

Patterns in the .dockerignore file are incorrectly following .gitignore format.
The main difference is that root directory / is implicit in .dockerignore format.
See https://docs.docker.com/engine/reference/builder/#dockerignore-file for reference.

This leads to not ignoring a bunch of files when building Docker images.
It can make Docker images to be larger than necessary or to contain unintended data.

Solution

  • Do not rely on defaults which are different from .gitignore syntax by explicitly prepending with / to match from the root directory only (default) or with **/ to match from any subdirectory (and from the root directory too).
  • Append an ending / to specify when ignoring directories
  • Extend some patterns to match more actual and potential files: .gitattributes, venv-py39

That pull request follows commit metabrainz/sir@c03d568 whihc addresses the same issue in SIR.

Action

  • Rebuild, check it still works for all build targets, and compare images size.
  • Possibly add more patterns to ignore more files. (I just fixed and extended existing patterns, I did not add any new pattern.)

Patterns are incorrectly following .gitignore format which is different.
The main change is root directory / is implicit in .dockerignore format.
See https://docs.docker.com/engine/reference/builder/#dockerignore-file

This patch fixes/extends existing patterns as follows:
* Adding a / in front of patterns matching subpaths of root directory.
  (Even though it is not required, it is less treacherous to git users.)
* Adding **/ in front of patterns matching subpaths of any directory.
  (This captures a lot more of *.pyc files and __pycache__/ dirs.)
Even though it is not strictly required, ending a path with a `/` makes
clear that only directories are supposed to be ignored by Docker build.
When creating a Docker image, skip all:
- `/.git*` such as .gitattributes .gitignore .github
- `/env*/` `/venv*/` `/build*/` such as venv-py39
@brainzbot
Copy link
Member

Can one of the admins verify this patch?

3 similar comments
@brainzbot
Copy link
Member

Can one of the admins verify this patch?

@brainzbot
Copy link
Member

Can one of the admins verify this patch?

@brainzbot
Copy link
Member

Can one of the admins verify this patch?

@yvanzo yvanzo changed the title Docker ignore Fix ignoring some files when building Docker images Jan 4, 2021
@alastair
Copy link
Collaborator

alastair commented Jan 4, 2021

@brainzbot add to whitelist

@alastair
Copy link
Collaborator

alastair commented Jan 4, 2021

So that I understand this better - the main issue is that something like

  • __pycache__/

only refers to a directory ./pycache in the root directory, and not in any other subdirectory? For this reason you've explicitly prefixed a / to the beginning of rules that refer to files in the root, and added **/ to rules that should apply in all subdirectories?

@yvanzo
Copy link
Contributor Author

yvanzo commented Jan 4, 2021

@alastair: Exactly, see also commits message and linked documentation.

It makes a difference mainly when building from a developer’s working local copy.

@yvanzo
Copy link
Contributor Author

yvanzo commented Jan 4, 2021

Note that **/ matches / too.

@alastair alastair merged commit ccf4bd3 into master Jan 12, 2021
@alastair alastair deleted the docker-ignore branch January 12, 2021 13:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants