get_url checksum does not support checksum files containing *just* the checksum

[DanHam]

SUMMARY

get_url fails to parse a checksum file when the file contains just the checksum

ISSUE TYPE

• Bug Report

COMPONENT NAME

get_url

ANSIBLE VERSION

ansible 2.8.0.dev0
  config file = /Users/dan/.ansible.cfg
  configured module search path = ['/Users/dan/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/dan/.venv-ansible/lib/python3.7/site-packages/ansible
  executable location = /Users/dan/.venv-ansible/bin/ansible
  python version = 3.7.2 (default, Dec 30 2018, 08:55:50) [Clang 10.0.0 (clang-1000.11.45.5)]

CONFIGURATION

HOST_KEY_CHECKING(/Users/dan/.ansible.cfg) = False
INVENTORY_ENABLED(/Users/dan/.ansible.cfg) = ['host_list', 'script', 'yaml', 'ini', 'auto', 'gcp_compute']

OS / ENVIRONMENT

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.3
BuildVersion:   18D109

STEPS TO REPRODUCE

- name: Demo error with plain checksum file
  get_url:
    checksum: sha512:https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler.sha512
    dest: /usr/bin/kube-scheduler
    mode: 0755
    url: https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler

For easy reference, the contents of the checksum file is as follows:

f400eedaef4aa277ba9ffbc17d1937fe2e200f5b4886930ae2c34e5a1a4ee14aee5d26c45b3babf7e791ba292787c950f55f4b9a32294472c73ec7b62af45858

EXPECTED RESULTS

get_url should be able to obtain the checksum from a file containing just the checksum.

At present get_url is able to handle the more complex case of parsing out the correct checksum from a file containing multiple checksums.

With the example shown above, there is no need to parse out the correct checksum. Here the checksum makes up the entirety of the file contents and all that is needed is to simply read the file.

ping @sivel as this was worked on recently in #53685

ACTUAL RESULTS

get_url fails and returns the following error

"msg": "The checksum parameter has to be in format <algorithm>:<checksum>"

[ansibot]

Files identified in the description:

• lib/ansible/modules/net_tools/basics/get_url.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @jpmens @ptux
click here for bot help

[sivel]

I think I would consider this a feature request.

In any case, for now you can do the following:

- name: Demo error with plain checksum file
  get_url:
    checksum: "sha512:{{ lookup('url', 'https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler.sha512') }}"
    dest: /usr/bin/kube-scheduler
    mode: 0755
    url: https://dl.k8s.io/v1.13.0/bin/linux/amd64/kube-scheduler

[DanHam]

@sivel OK. In the meantime it would be great if the docs could specify exactly what the expected format of the checksum file should be - this one had me scratching my head for a while as I just assumed this would be supported.

Many thanks for the workaround in the interim.

[sivel]

I've had a few minutes to play with a solution, but haven't tested it extensively.

diff --git a/lib/ansible/modules/net_tools/basics/get_url.py b/lib/ansible/modules/net_tools/basics/get_url.py
index ea033d9cfe..d25ae0c39c 100644
--- a/lib/ansible/modules/net_tools/basics/get_url.py
+++ b/lib/ansible/modules/net_tools/basics/get_url.py
@@ -495,12 +495,19 @@ def main():
             with open(checksum_tmpsrc) as f:
                 lines = [line.rstrip('\n') for line in f]
             os.remove(checksum_tmpsrc)
-            checksum_map = {}
-            for line in lines:
-                parts = line.split(None, 1)
-                if len(parts) == 2:
-                    checksum_map[parts[0]] = parts[1]
             filename = url_filename(url)
+            checksum_map = {}
+            if len(lines) == 1 and len(lines[0].split()) == 1:
+                # Only a single line with a single string
+                # treat it as a checksum only file
+                checksum_map[lines[0]] = filename
+            else:
+                # The assumption here is the file is in the format of
+                # filename checksum
+                for line in lines:
+                    parts = line.split(None, 1)
+                    if len(parts) == 2:
+                        checksum_map[parts[0]] = parts[1]
 
             # Look through each line in the checksum file for a hash corresponding to
             # the filename in the url, returning the first hash that is found.

If anyone wants to take that patch, and get together a PR with a changelog fragment, and tests, please feel free to do so. I am not actively pursuing this further.

[ansibot]

Files identified in the description:

• lib/ansible/modules/get_url.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[muellert]

It would be great if the module's documentation would specify the format the checksum file needs to be in, and if one could have more than one checkum in one file, for a number of algorithms or files. So far, I tried "xxxxx" (the sha256sum checksum), "xxxxx filename", "sha256:xxxxx" and also "SHA256:xxxxx filename", all without success.