Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Ignores from lower layers can cause higher layer files to be ignored (dropped) #220
I've found that not all files survive the charm build process. In this example, the layer I am building loses its README.md file from the "top" layer.
Take note: README.md is here.
Take note: README.md did not survive to the built artifact.
Versions (when using charm build)
Versions (when using tox + venv)
FWIW, I see the same when using tox + venv + pip:
I expect/expected the following
Top layer root directory files should survive to the built charm.
What I got
Lost a file.
One of the lower layers  declares README.md ignored. The expected behavior is that the README.md file from only that layer is ignored. But it appears to be affecting even the top layer.
To check if this is resolved by d31304e, I re-built with requirements defined as `git+https://github.com/juju/charm-tools#egg=charm-tools`` (to pull in the committed fix from #218), but no love. The lower ignore prevails above still.
Just to confirm, base_layer.fetch() from that patch is present:
The README.md file is still dropped:
This is because a layer under the top layer has an
ignore: - 'README.md'
While the easy solution is to just remove that line, the ability to do an ignore per layer instead of globally is absolutely necessary. In addition, being able to ignore files from the top layer in lower layers seems equally as important. I'd like to propose the following.
First, that a new
layer-ignore: layer-name: - 'FILE' - 'path/file' other-layer: - 'path'
The second change is that the current ignore key:
ignore: - 'README.md'
would be transformed to
layer-ignore: layer-name: - 'README.md'
Although we worked around the one instance of this issue, it is still a blocker to certain aspects of our development in that we really do need to exclude unit_tests and other files from lower layers.
With the current "global" ignore behavior, the ignore feature is dangerous to use and difficult to troubleshoot because any arbitrary layer can cause any other layer's files to be dropped at build time.
I think the direction this is headed is a good one.
The model is that for each layer from base to final we construct a set of files that will be in the final result. The base layer needs a way to say, "while this is in my repo don't include it in the set"
and the consumer (the layer above it) needs a way to say, I know I don't want this file from the base.
After that you are left with the set of all files except the ones the base didn't promote and the one that the consumer ignored.
If there is a layer above that one take the existing set and apply the rules again. The difference from today would be clearing the ignores like after each layer is processed and the addition of exclude, saying don't promote these files or directories.
How does this sound?