Community package requirements: sanity tests

[gotmax23]

Community package requirements: sanity tests

(Note: This issue was filed in a semi-automated fashion. Let me know if you see errors in this issue.)

As per the Ansible community package inclusion requirements, collections must pass ansible-test sanity tests. Version 1.10.0 of cisco.ucs, corresponding to the v1.10.0 tag in this repo, fails one or more of the required sanity tests.

Please see the errors below and address them. If these issues aren't addressed within a reasonable time period, the collection may be subject to removal from Ansible.

Thank you for your efforts and for being part of the Ansible package! We appreciate it.

---

Sanity tests

The following tests were run using ansible-test version 2.16.1:

• ansible-doc
• compile
• validate-modules
• yamllint

Note that this is only a subset of the required sanity tests. Please make sure you run them in all in your CI.

Results

The test ansible-test sanity --test validate-modules [explain] failed with 9502 errors:

Issue truncated. Click to read more.

[dsoper2]

There looks to be an issue in the sanity test output. All modules are reporting the same errors:
plugins/modules/ucs_dns_server.py:0:0: doc-choices-do-not-match-spec: Argument 'access_policy' in argument_spec found in virtual_drive defines choices as (['blocked', 'hidden', 'platform-default', 'read-only', 'read-write', 'transport-ready']) but documentation defines choices as ([])
virtual_drive related code is only in the ucs_disk_group_policy module and it passes sanity tests without error:
ansible-test sanity --test validate-modules plugins/modules/ucs_disk_group_policy.py
Running sanity test "validate-modules"

[gotmax23]

Do you have a tests/sanity directory with ignore entries that are not included in this repository? We are running tests from your upstream git repository so if you have extra files included in an internal repo or elsewhere, we cannot account for those.

EDIT: Or maybe there's some other difference to Automation Hub's sanity tests that I'm not aware of. I'll ask someone on the Steering Committee who's more familiar with AH than I to take a look. I know you mentioned that AH's sanity tests were passing just fine in CiscoDevNet/intersight-ansible#108.

[felixfontein]

@dsoper2 there is no issue with the sanity test output. The validate-modules test fails for this collection with a very long list of failures, both with ansible-core devel and ansible-core 2.17.

Automation Hub likely only runs a more lenient version of the sanity tests. Passing them is nice, but no indication that all sanity tests pass.

Another problem with this repository is that there doesn't seem to be any CI in this repository. The inclusion requirements are pretty clear on this:

• All CI tests MUST run against every pull request and SHOULD pass before merge.
• At least sanity tests MUST run against a commit that releases the collection; if they do not pass, the collection will NOT be released.
• All CI tests MUST run regularly (nightly, or at least once per week) to ensure that repositories without regular commits are tested against the latest version of ansible-test from each ansible-core version tested. The results from the regular CI runs MUST be checked regularly.

(https://docs.ansible.com/ansible/devel/community/collection_contributors/collection_requirements.html#ci-testing)

This does not seem to be the case for this repository. This is a clear violation of the inclusion requirements.

[gotmax23]

This collection seems to have the issue that Felix mentioned in CiscoDevNet/intersight-ansible#108 where multiple modules modify a global dict in a different way. Making a copy of the dict each time should solve that.

[dsoper2]

CI added and ansible-test sanity now passing