Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
get_url does not handle CHECKSUMS files from URL unless the file contains a single checksum #48790
get_url seems to only handles a downloaded checksums file if that file contains a single checksum.
OS / ENVIRONMENT
STEPS TO REPRODUCE
Point at a project's CHECKSUMS file URL so that I avoid downloading the file if I already have a known good copy locally.
If the checksums file is gpg signed, I get
If I put the checksums file on a ftp server of mine and remove the gpg parts, I get
Only if I reduce the checksums file to a single line I get success.
Now obviously I normally have no control over an upstream checksums file and would really like to be able to use one like for example at https://releases.ansible.com/ansible-tower/setup-bundle/ansible-tower-setup-bundle-CHECKSUM as most checksums files I use when manually checking are in that format (gpg signed an with multiple checksums).
Note: Alberto had a look and found this;
We're not going to add knowledge of formats for checksum files to get_url. get_url's parameter command is for passing in a checksum, not for extracting a checksum from a file.
The playbook author should have the knowledge of the checksum file's formats and can use jinja2 to extract the checksum from an upstream file if they wish. For example, extracting the checksum for the tower bundled installer can be done like this:
We did eventually find that the conclusion was incorrect about only supporting a single checksum in the file.
There was overzealous exception handling that was obscuring the actual issue.
As such, this has been fixed in #53685 and a backport for 2.7 has been submitted as well.
In my opinion checksum parsing should be implemented as a part of Ansible core for modules to reuse it. Checksum can be defined in-place(string/variable value), URL (http[s]?://,ftp://) or in file (file://) and has different format:
Nowadays majority of popular distros provide sha256sum.txt file (Ubuntu/Debian/Centos to name some of them), same applies to cloud providers (Amazon VM imges).
Workflow "register/lookup/jinja2" looks extremely awful and error prone.
* [stable-2.7] Fix checksum file parsing in get_url (#53685) * Fix checksum file parsing. Fixes #48790 * guard invalid int conversion Co-Authored-By: sivel <firstname.lastname@example.org> * Remove extra newline. (cherry picked from commit 77217fd) Co-authored-by: Matt Martz <email@example.com> * Remove use of undefined variable
@sivel Unfortunately get_url (even with the fix implemented in #53685) doesn't support the simplest case where a checksum file contains just the checksum. Clearly, in this instance, there is no need to parse out the correct checksum. Here the checksum makes up the entirety of the file contents and all that is needed is to simply read the file.
Simplest example that demonstrates this:
- name: Demo error with plain checksum file get_url: checksum: sha512:https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler.sha512 dest: /usr/bin/kube-scheduler mode: 0755 url: https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler.sha512
The error is:
For reference, the contents of the checksum file is as follows:
Clearly, it's fairly trivial to work around this as per #48790 (comment). However, if get_url can handle the more difficult case of parsing out a checksum from a file, it would be great if it is able to handle the simplest of cases as well.