Fix issues found while importing collection to Automation Hub

[geerlingguy]

SUMMARY

Update: See this comment for the current issues.

When uploading version 1.0.0 of the downstream redhat.openshift collection, we ran into a few errors.

Error running ansible-doc: cmd="/usr/bin/env ANSIBLE_COLLECTIONS_PATHS=/tmp/tmp58gaersy ansible-doc --type module --json redhat.openshift.openshift_route redhat.openshift.openshift_auth redhat.openshift.k8s redhat.openshift.k8s_auth redhat.openshift.openshift_process" returncode="1" b"[WARNING]: Failed to create the directory '/.ansible': [Errno 13] Permission\ndenied: b'/.ansible'\nERROR! module redhat.openshift.openshift_route missing documentation (or could not parse documentation): unknown doc_fragment(s) in file /tmp/tmp58gaersy/ansible_collections/redhat/openshift/plugins/modules/openshift_route.py: kubernetes.core.k8s_auth_options, kubernetes.core.k8s_wait_options, kubernetes.core.k8s_state_options\n" 
...
ERROR: Found 1 shebang issue(s) which need to be resolved: 
ERROR: ci/incluster_integration.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/shebang.html 
Running sanity test 'shellcheck' 
ERROR: Found 8 shellcheck issue(s) which need to be resolved: 
ERROR: ci/downstream.sh:35:21: SC2068: Double quote array expansions to avoid re-splitting elements. 
ERROR: ci/incluster_integration.sh:14:1: SC2034: component appears unused. Verify it or export it. 
ERROR: ci/incluster_integration.sh:15:12: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:19:23: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:21:23: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:24:12: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:55:82: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:63:80: SC2086: Double quote to prevent globbing and word splitting. 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/shellcheck.html 

We should also see if we can get the errors to reproduce in our own automated tests so we don't run into similar errors in the future.

ALSO, I just noticed the repo that ends up being referenced is https://github.com/ansible-collections/redhat.openshift, and the issue tracker is https://github.com/ansible-collections/redhat.openshift/issues — neither of which currently exist.

ISSUE TYPE

• Bug Report

COMPONENT NAME

Automation Hub downstream release.

Related: #51

See release log: https://cloud.redhat.com/ansible/automation-hub/redhat/openshift/import-log?version=1.0.0 (requires authentication).

[geerlingguy]

@tima - Since https://github.com/ansible-collections/redhat.openshift doesn't exist—which of these options should we choose:

1. Create that repo, but add a skeleton to it that just says "This is a placeholder, the source for the redhat.openshift collection is in community.okd."
2. Update our build scripts to not change the repo and issue tracker values inside galaxy.yml so Automation Hub points straight at the community project on GitHub.

[geerlingguy]

For the ansible-doc issue... I wonder if Automation Hub supports cross-collection documentation fragments? I know we are pulling in fragments from kubernetes.core, and maybe that's causing the ansible-doc failure?

[geerlingguy]

I created a bug in the Automation Hub Jira tracker here: https://issues.redhat.com/browse/AAH-154

[geerlingguy]

It looks like the downstream build script has a line:

find ${_build_dir} -type f -exec sed -i.bak "s/community\.kubernetes/kubernetes\.core/g" {} \;

And that seems like it should take care of at least most of the dependency issues on Automation Hub. But it would still be better to just fix up this collection to depend on kubernetes.core in the first place.

[geerlingguy]

Created PR #58.

[geerlingguy]

The documentation issue should be fixed on next release by this PR: #59

However, there could still be some lingering issues to clean up, I'll update this issue once 1.0.1 is released and see what the import log shows.

[geerlingguy]

Current issues in the AH import (docs issue is resolved, though, yay!):

Running sanity test 'future-import-boilerplate' with Python 3.6 
ERROR: Found 1 future-import-boilerplate issue(s) which need to be resolved: 
ERROR: ci/downstream_fragments.py:0:0: missing: from __future__ import (absolute_import, division, print_function) 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/future-import-boilerplate.html 
...
Running sanity test 'metaclass-boilerplate' with Python 3.6 
ERROR: Found 1 metaclass-boilerplate issue(s) which need to be resolved: 
ERROR: ci/downstream_fragments.py:0:0: missing: __metaclass__ = type 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/metaclass-boilerplate.html 
...
Running sanity test 'pep8' with Python 3.6 
ERROR: Found 1 pep8 issue(s) which need to be resolved: 
ERROR: ci/downstream_fragments.py:26:1: W391: blank line at end of file 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/pep8.html 
Running sanity test 'pylint' with Python 3.6 
ERROR: Found 1 pylint issue(s) which need to be resolved: 
ERROR: ci/downstream_fragments.py:26:0: trailing-newlines: Trailing newlines 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/pylint.html 
...
Running sanity test 'shebang' with Python 3.6 
ERROR: Found 2 shebang issue(s) which need to be resolved: 
ERROR: ci/downstream.sh:1:1: unexpected non-module shebang: b'#!/bin/bash -e' 
ERROR: ci/incluster_integration.sh:1:1: unexpected non-module shebang: b'#!/bin/bash' 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/shebang.html 
...
Running sanity test 'shellcheck' 
ERROR: Found 10 shellcheck issue(s) which need to be resolved: 
ERROR: ci/downstream.sh:35:21: SC2068: Double quote array expansions to avoid re-splitting elements. 
ERROR: ci/downstream.sh:158:13: SC2097: This assignment is only seen by the forked process. 
ERROR: ci/downstream.sh:159:40: SC2098: This expansion will not see the mentioned assignment. 
ERROR: ci/incluster_integration.sh:14:1: SC2034: component appears unused. Verify it or export it. 
ERROR: ci/incluster_integration.sh:15:12: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:19:23: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:21:23: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:24:12: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:55:82: SC2086: Double quote to prevent globbing and word splitting. 
ERROR: ci/incluster_integration.sh:63:80: SC2086: Double quote to prevent globbing and word splitting. 
See documentation for help: https://docs.ansible.com/ansible/2.9/dev_guide/testing/sanity/shellcheck.html

Import log (might need login to view): https://cloud.redhat.com/ansible/automation-hub/redhat/openshift/import-log?version=1.0.1

[Akasurde]

@geerlingguy #63 resolves all the issues mentioned. Could you please verify this while you import the new release? Thanks,

[geerlingguy]

@Akasurde - If I pull main and run make downstream-test-sanity, I am still getting one more error:

ERROR: Found 1 validate-modules issue(s) which need to be resolved:
ERROR: plugins/modules/k8s_auth.py:0:0: invalid-documentation: DOCUMENTATION.module: not a valid value for dictionary value @ data['module']. Got 'openshift_auth'

Just noting that what I see in the k8s_auth.py module file is:

#!/usr/bin/python
# -*- coding: utf-8 -*-

# Copyright (c) 2018, KubeVirt Team <@kubevirt>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)

from __future__ import absolute_import, division, print_function
__metaclass__ = type

DOCUMENTATION = r'''

module: openshift_auth

short_description: Authenticate to OpenShift clusters which require an explicit login step

version_added: "0.2.0"

...