Separate downloading and unpacking prometheus archive

[paulfantom]

Resolves #101

Possibly accidentally solves ans issue with too many downloads of prometheus archive to localhost.

[taiansu]

Thanks for the blasting fast fixing. I tried the fix, and with two minor modifications, it works correctly.

The changes are:

1. Replace line#55 's delegate_to: localhost with remote_src: yes
2. Add a remote_src: yes for copy module, for example, between line#61 and line#62.

[SuperQ]

Thinking about this, can we download these archives to something like /opt/prometheus/.archive/ rather than /tmp? I'm not sure if the ansible get_url is safe against existing symlinks/hardlinks in /tmp.

[paulfantom]

@taiansu your modifications would change how we want this role to work. If we add remote_src in the places you say it will cause ansible to download archives to target hosts and this is highly inefficient in large infrastructures (also sometimes won't work if target hosts doesn't have internet access).
Our approach is to download archives from internet to deployer machine (one which runs ansible) and propaagte from this machine to other nodes.

[paulfantom]

@SuperQ we could probably use tmp_dest feature of get_url module to solve this issue, but I need to look closer at it.

[paulfantom]

@taiansu can you tell what is not working in current version? Some error messages could be useful.

[SuperQ]

One other nice thing, it would be good to cache this so it doesn't download the file every time. In theory, we could extract the checksum from the sums published on github.

[paulfantom]

@SuperQ I agree that it would be nice to validate checksums. This option is even provided in get_url module. However first we would need to download checksums. I think it would be better to split it into another issue (#104).
As for caching, this won't be possible with tmp_dest parameter and I don't think that storing it in /opt is a good idea. Currently we provide some caching functionality by storing files in /tmp directory, which is cleaned only after deployer machine is rebooted.

[taiansu]

Yes I understand that will change the behaviour. If I apply the changes as is, the first error I got is like:

fatal: [test -> localhost]: FAILED! => {"changed": false, "msg": "Failed to get information on remote file (/tmp): sudo: a password is required\n"}

And adding become: yes to the unarchive module still got the same error. Then this leads me to the thread ansible/ansible#6131 (comment) which describes the presumption of unarchive module.

[paulfantom]

@taiansu this should be fixed now, could you test it?

[paulfantom]

@SuperQ I don't think we can use other directories on deployer machine apart from /tmp due to permission issues.

[rdemachkovych]

For me, it's required install libselinux-python and work fine

[taiansu]

I tried the fix, but I got the simillar error back to the first point:

fatal: [test -> localhost]: FAILED! => {"changed": false, "msg": "Failed to find handler for \"/Users/tai/.ansible/tmp/ansible-tmp-1524074379.55-78415089495321/source\". Make sure the required command to extract the file is installed. Command \"/usr/bin/tar\" detected as tar type bsd. GNU tar required. Command \"/usr/bin/unzip\" could not handle archive."}

So I'm wondering the error may not be relate to the separate the download and unarchve, but rather we just can't unarchive a linux tar in the host while it's a Mac.

Anyway thanks for all the help.

[paulfantom]

@taiansu yes it seems that there is a problem with tar on Mac:

Make sure the required command to extract the file is installed. Command "/usr/bin/tar" detected as tar type bsd. GNU tar required. Command "/usr/bin/unzip" could not handle archive.

Could you check if after installing gnu-tar this works?

brew install gnu-tar

[taiansu]

It works. Thanks so much!

[paulfantom]

Great!
I've added this to readme in requirements section.

[rdemachkovych]

Should we also update the Requiremets in the README?

[rdemachkovych]

LGTM

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.