change docker image build path

[alxgomz]

PR intend to fix #50 by changing the build path to either:

• a path specified in the molecule.yml file
• the scenario directory where molecule looks for custom Dockerfile.j2 (so path specified in COPY instruction remain valid)

There may be implications I didn't think of but from the basic tests I've made "it just work on my laptop" :)

[ssbarnea]

While I do understand that you want to make it easy to use COPY command I am bit against this move for a different reason: you endup producing temp files (dockerfile) inside the source tree.

Testing framework should not produce leftovers (untracked or modified) files after running and this change modifies that.

I think that the correct approach for this is to template paths used inside COPY commands when you want to do that.

Maybe someone has a better idea about how to address that aspect.

[alxgomz]

I may be misunderstanding but I can't think of a way to use templating in COPY instructions within a Dockerfile that would help.
Correct me if I'm wrong but the files to COPY will need to live within the docker build path...

Would that be more acceptable if there were a cleanup playbook to deal with that leftover Dockerfiles?

[ssbarnea]

As the Dockerfile.j2 is a jinja2 template I think it should be possible to use MOLECULE_SCENARIO_DIRECTORY inside. If I remember correctly it should look like {{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}

[tadeboro]

I am a bit biased here because I really dislike the "let me build the image right before the test"-parts of the molecule. For all our tests, we maintain our own docker images that molecule then consumes directly.

That being said, I do see how the COPY command not working might be annoying for users that do use the ability to build images on the fly. And having to template the COPY parameters is just as annoying (and will probably fail in quite a lot of scenarios because IIRC, COPY can only access files in the context used at image build time).

One "clean" solution I see here is to copy the whole scenario directory into some temporary location, build the image there, and then delete the whole thing. Building images in the scenario directory is another alternative that has the potential to leave the temporary files in the project folder, which is not something I would like to see. I already hate the molecule for creating the .cache directory behind my back.

[alxgomz]

As the Dockerfile.j2 is a jinja2 template I think it should be possible to use MOLECULE_SCENARIO_DIRECTORY inside. If I remember correctly it should look like {{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}

Sure, but then the files to COPY would live in the scenario directory, while the docker build is done using MOLECULE_EPHEMERAL_DIRECTORY as build path so the COPY would not work (as explained by @tadeboro). That's what I tried to explain in my previous comment but was maybe unclear

[alxgomz]

I realize having a new ootb playbook might create concerns but here is an example cleanup I had in mind (didn't test it):

---
- name: Cleanup
  hosts: localhost
  connection: local
  gather_facts: false
  no_log: "{{ molecule_no_log }}"
  vars:
    molecule_labels:
      owner: molecule
  tasks:
    - name: Cleanup Dockerfiles 
      file:
        state: absent
        path: "{{ item.path | default(molecule_scenario_directory) }}/Dockerfile_{{ item.image | regex_replace('[^a-zA-Z0-9_]', '_') }}"
      when: item.path

[ssbarnea]

Cleanup is no always called, especially when doing development. This means that relying on it for clearing tmp files is not really the best idea.

Have you tried to use the templating directly into the Dockerfile and to avoid changing the directory where we put it? I am confident that this would work. If it works an PR would still be useful that would exemplify how to use COPY with that path, so other users will know how to write Dockerfiles that work well with molecule.

[alxgomz]

Just did the test and it fails as I expected due to the file being outside of the build context:

    TASK [Build an Ansible compatible image (new)] *********************************
    FAILED - RETRYING: Build an Ansible compatible image (new) (3 retries left).
    FAILED - RETRYING: Build an Ansible compatible image (new) (2 retries left).
    FAILED - RETRYING: Build an Ansible compatible image (new) (1 retries left).
    failed: [localhost] (item=molecule_local/docker.io/pycontribs/centos:8) => {"ansible_loop_var": "item", "attempts": 3, "changed": false, "item": {"ansible_index_var": "i", "ansible_loop_var": "item", "changed": true, "checksum": "b41ed4f9d7df3c294e47d02ea84bf4af843d53f0", "dest": "/home/alxgomz/.cache/molecule/test1/default/Dockerfile_docker_io_pycontribs_centos_8", "diff": [], "failed": false, "gid": 1000, "group": "alxgomz", "i": 0, "invocation": {"module_args": {"_original_basename": "Dockerfile.j2", "attributes": null, "backup": false, "checksum": "b41ed4f9d7df3c294e47d02ea84bf4af843d53f0", "content": null, "delimiter": null, "dest": "/home/alxgomz/.cache/molecule/test1/default/Dockerfile_docker_io_pycontribs_centos_8", "directory_mode": null, "follow": false, "force": true, "group": null, "local_follow": null, "mode": "0600", "owner": null, "regexp": null, "remote_src": null, "selevel": null, "serole": null, "setype": null, "seuser": null, "src": "/home/alxgomz/.ansible/tmp/ansible-tmp-1620834848.9412036-330782-203167379580750/source", "unsafe_writes": null, "validate": null}}, "item": {"image": "docker.io/pycontribs/centos:8", "name": "instance", "pre_build_image": false}, "md5sum": "ed14df0941d4a6698bfbba660cc45f63", "mode": "0600", "owner": "alxgomz", "size": 87, "src": "/home/alxgomz/.ansible/tmp/ansible-tmp-1620834848.9412036-330782-203167379580750/source", "state": "file", "uid": 1000}, "msg": "Error building molecule_local/docker.io/pycontribs/centos - code: None, message: COPY failed: file not found in build context or excluded by .dockerignore: stat tmp/test/test1/molecule/default/test.txt: file does not exist, logs: ['Step 1/2 : FROM docker.io/pycontribs/centos:8', '\\n', ' ---> fec8cb567d93\\n', 'Step 2/2 : COPY /tmp/test/test1/molecule/default/test.txt /tmp', '\\n']"}

here is the molecule.yml:

---
dependency:
  name: galaxy
driver:
  name: docker
platforms:
  - name: instance
    image: docker.io/pycontribs/centos:8
    pre_build_image: false
provisioner:
  name: ansible
verifier:
  name: ansible

And the Dockerfile.j2:

FROM {{ item.image }}
COPY {{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}/test.txt /tmp

Sanity check:

cat /tmp/test/test1/molecule/default/test.txt
test

[ssbarnea]

@alxgomz Can you please show the generated Dockerfile made by molecule, or at least extract the COPY line from it? I am curious what went wrong. I was not able to find any place that states that you cannot use COPY with full paths.

Do you have any custom .dockerignore file that lists /tmp? I also found https://jhooq.com/docker-copy-failed-no-source-files-were-specified/ article which may help finding the source of the error.

I wonder if the error may be caused by the weird destination instead of the source. Maybe you could try to copy to /root just to see if it does work.

[tadeboro]

@ssbarnea From the docker docs at https://docs.docker.com/engine/reference/builder/#copy:

Multiple resources may be specified but the paths of files and directories will be interpreted as relative to the source of the context of the build.

So templating the COPY parameters will not work most of the time. This is why I proposed the "copy the whole scenario directory"-approach. In this case, even if we abort the operation in a really nasty way that does no cleanup whatsoever, the worst we did is clutter the temporary directory a bit.

[alxgomz]

below is what the template expanded to:

FROM docker.io/pycontribs/centos:8
COPY /tmp/test/test1/molecule/default/test.txt /tmp

No .dockerignore. The key really is with the fact files need to live within the build path. You can easily double check it by trying manually building an image usingdocker CLI only. For example using COPY ../filetocopy /tmp will failed whatsoever.
It actually fails with a very explicit message:

docker build . -t test
Sending build context to Docker daemon  8.704kB
Step 1/2 : FROM docker.io/pycontribs/centos:8
 ---> fec8cb567d93
Step 2/2 : COPY ../test.txt /tmp
COPY failed: forbidden path outside the build context: ../test.txt ()

This is what @tadeboro pointed earlier today.

[tadeboro]

I came up with xlab-steampunk@7cd45f2 as an alternative way of solving this issue. That additional task copies over the contents of the scenario directory, which should allow users to write their Dockerfiles as if the build will be executed from the scenario directory.

I did not test this because it is really late here. @alxgomz, will you be so kind and test this alternative approach and make sure it solves your problem? Thank you in advance.

[alxgomz]

I was not super keen of copying files in a bulk, but I have to say that it worked quite nicely for me.
I'm just wondering if there is a chance this copy approach would conflict with some other logic in molecule itself? (e.g. over writing files when doing our copy).
Other than that, I can't find other concrete reasons not to use that approach (I think @ssbarnea seems to prefer that approach too)
@tadeboro do you plan to make it a PR?

[alxgomz]

Thinking twice, i can see one down side in using file copy without setting the build path. I have a playbook with multiple roles, and they all use the same Dockerfile.j2 file and the same filetoCopy within it.
By not being able to set the docker_image build path, the filetoCopy (used by COPY instructions) needs to be in the scenario directory which is per role. So one would need to duplicate that file in each role thus creating multiple identical copies of the same files to maintain accross the project... :-/

[tadeboro]

Thinking twice, i can see one down side in using file copy without setting the build path. I have a playbook with multiple roles, and they all use the same Dockerfile.j2 file and the same filetoCopy within it.
By not being able to set the docker_image build path, the filetoCopy (used by COPY instructions) needs to be in the scenario directory which is per role. So one would need to duplicate that file in each role thus creating multiple identical copies of the same files to maintain accross the project... :-/

I would say that this use case can be solved in a cleaner way by using prebuilt docker images. Just build the image before running the molecule and save time when running scenarios. I do see the value of being able to build scenario-specific images on the fly, but image that is shared across different scenarios feels like something that is out of scope for molecule.

[tadeboro]

I'm just wondering if there is a chance this copy approach would conflict with some other logic in molecule itself? (e.g. over writing files when doing our copy).

The commit I created is just a proof-of-concept implementation. We can always add some validation to make sure we do not override anything important or move the whole scenario directory into a sub dir in the ephemeral directory. I just wanted to see if the alternative implementation would work.

@tadeboro do you plan to make it a PR?

I was hoping that if we decide that this is the way we want to go, you will create a new one or update the existing one ;)

[alxgomz]

Ah just seeing your comment now after I pushed an update to my PR.
Still not ideal, but I tried to stick to current behaviour as much as possible so with this one, if user wants to use COPY he can and everything will be built in MOLECULE_EPHEMERAL_DIRECTORY, unless the path option is specified in molecule.yml, then the build would happen where user wants it to happen. In that later case - or of something goes wrong - the cleanup task can take care of removing the generated Dockerfile.

Whichever method is chosen it would require a bit of documentation, which I'm not sure where would be best suited.

@tadeboro no worries, I'll take care of the PR once we know what is the right way forward (and thanks for helping out here).

[tadeboro]

Do we know what approach we will take? Because right now, the changes in this PR implement both of them as far as I can tell.