gcp_storage_object fails with service_account_contents option

[ralbon]

SUMMARY

When I use service_account_contents option, the module call fails.
When adding a debug task with "{{ lookup('file', gitlab_sa_key_path) }}"the json content is properly displayed.

ISSUE TYPE

• Bug Report

COMPONENT NAME

gcp_storage_object.py

ANSIBLE VERSION

ansible 2.8.0rc2
  config file = None
  configured module search path = ['/Users/romain.albon/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/romain.albon/.virtualenvs/gcp-tools/lib/python3.7/site-packages/ansible
  executable location = /Users/romain.albon/.virtualenvs/gcp-tools/bin/ansible
  python version = 3.7.1 (default, Nov  6 2018, 18:46:03) [Clang 10.0.0 (clang-1000.11.45.5)]

CONFIGURATION

ANSIBLE_FORCE_COLOR(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = -F ssh_config
DEFAULT_REMOTE_USER(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = ansible
DEFAULT_ROLES_PATH(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = ['/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/roles-external', '/Us
DEFAULT_VAULT_PASSWORD_FILE(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = /Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible_vault_
HOST_KEY_CHECKING(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = False
RETRY_FILES_ENABLED(/Users/romain.albon/Documents/Projects/FnacDarty/gcp-tools/ansible/ansible.cfg) = False

OS / ENVIRONMENT

N/A

STEPS TO REPRODUCE

- name: Trigger gitlab upload of gitlab-secrets file
  gcp_storage_object:
    action: upload
    bucket: "{{ gitlab_backup_gcs_name }}"
    src: /etc/gitlab/gitlab-secrets.json
    dest: "{{ backup_id.stdout }}_gitlab-secrets.json"
    auth_kind: "serviceaccount"
    service_account_contents: "{{ lookup('file', gitlab_sa_key_path) }}"

EXPECTED RESULTS

File uploaded

ACTUAL RESULTS

fatal: [tools-gitlab-dev]: FAILED! => {
    "changed": false,
    "module_stderr": "Warning: Permanently added 'XX.XXX.113.232' (ECDSA) to the list of known hosts.\r\nWarning: Permanently added 'XXX' (ECDSA) to the list of known hosts.\r\nConnection to XXX closed.\r\n",
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1557410508.844717-145048655204206/AnsiballZ_gcp_storage_object.py\", line 114, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1557410508.844717-145048655204206/AnsiballZ_gcp_storage_object.py\", line 106, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1557410508.844717-145048655204206/AnsiballZ_gcp_storage_object.py\", line 49, in invoke_module\r\n    imp.load_module('__main__', mod, module, MOD_DESC)\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/__main__.py\", line 284, in <module>\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/__main__.py\", line 154, in main\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/__main__.py\", line 224, in fetch_resource\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/ansible_gcp_storage_object_payload.zip/ansible/module_utils/gcp_utils.py\", line 79, in get\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/ansible_gcp_storage_object_payload.zip/ansible/module_utils/gcp_utils.py\", line 126, in session\r\n  File \"/tmp/ansible_gcp_storage_object_payload_n1xrc9/ansible_gcp_storage_object_payload.zip/ansible/module_utils/gcp_utils.py\", line 155, in _credentials\r\n  File \"/usr/lib/python2.7/json/__init__.py\", line 339, in loads\r\n    return _default_decoder.decode(s)\r\n  File \"/usr/lib/python2.7/json/decoder.py\", line 364, in decode\r\n    obj, end = self.raw_decode(s, idx=_w(s, 0).end())\r\n  File \"/usr/lib/python2.7/json/decoder.py\", line 380, in raw_decode\r\n    obj, end = self.scan_once(s, idx)\r\nValueError: Expecting property name: line 1 column 2 (char 1)\r\n",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}

[ansibot]

Files identified in the description:

• lib/ansible/modules/cloud/google/gcp_storage_object.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @erjohnso @GoogleCloudPlatform @rambleraptor
click here for bot help

[barmesh]

@ralbon this is what I've found in the module docs:

This should not be passed in as a dictionary, but a string that has the exact contents of a service account json file (valid JSON)

As the docs said, you should pass the file content as a string:

- name: Trigger gitlab upload of gitlab-secrets file
  gcp_storage_object:
    action: upload
    bucket: "{{ gitlab_backup_gcs_name }}"
    src: /etc/gitlab/gitlab-secrets.json
    dest: "{{ backup_id.stdout }}_gitlab-secrets.json"
    auth_kind: "serviceaccount"
    service_account_contents: "{{ lookup('file', gitlab_sa_key_path) | string }}"

[Akasurde]

@ralbon I agree with @barmesh's comment. I raised PR #59427 to handle exception raised when service_account_contents is not converted to json.

[Akasurde]

resolved_by_pr #59427