ansible-test should warn when all collections content is ignored by git

[anshulbehl]

SUMMARY

Running ansible-test with collections inside a folder which is ignored by git will result is all targets skipped by ansible-test.

ISSUE TYPE

Feature Idea

COMPONENT NAME

ansible-test

ANSIBLE VERSION

ansible 2.9.0.dev0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/abehl/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/abehl/work/src/anshul_ansible/ansible/lib/ansible
  executable location = /home/abehl/work/src/anshul_ansible/ansible/bin/ansible
  python version = 2.7.16 (default, Apr 30 2019, 15:54:43) [GCC 9.0.1 20190312 (Red Hat 9.0.1-0.10)]

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

Run ansible test inside a collection that is in a folder which is ignored by git

EXPECTED RESULTS

Some warning should be displayed

ACTUAL RESULTS

No warning or way to know what is wrong

[ansibot]

Files identified in the description:

• bin/ansible-test

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[AlanCoding]

I also hit this pretty hard, I was going to file a new issue but this one covers it, so I'll give my writeup here.

Situation: I need to get the sanity tests to run, ansible-test sanity --test validate-modules will probably cover it. Because we can't symlink and run the tests, and because of the needed directory structure, I copy files before running the command in a Makefile target. That looks mostly the same as #60215 (comment). But unlike that example, I plan to run this locally a lot, so I put the adjacent folder (ansible_collections there, sanity in my case) in the .gitignore. That creates problems which require even more hacks.

I have most of what's needed up at ansible/awx#5004. See this target for the overall flow:

test_collection_sanity:
	rm -rf sanity
	mkdir -p sanity
	mkdir -p sanity/ansible_collections
	mkdir -p sanity/ansible_collections/awx
	cp -Ra awx_collection sanity/ansible_collections/awx/awx  # symlinks do not work
	cd sanity/ansible_collections/awx/awx && ansible-playbook -i localhost, make_imports_absolute.yml  # hack because sanity tests do not fully work
	cd sanity/ansible_collections/awx/awx && git init && git add . # requires both this file structure and a git repo, so there you go
	cd sanity/ansible_collections/awx/awx && ansible-test sanity --test validate-modules

(hack comment is in reference to #61884)

To explain the need for git init etc, consider that git ls-files returns no content in this folder because it is ignored by the .gitignore. That is the mechanical reason it gives the "All targets skipped" warning/error.

I really feel needs some solution. Requiring the file structure ansible_collections/foo/bar is reasonable by itself (knowing the python path reasons for it). But requiring that the files be in source control is a diametrically conflicting requirement.

There are 2 sides the the coin. The testing should require either:

• the collection should exist as it is from ansible-galaxy collection install and ready to be used in a playbook
• the collection should exist as it is in source, ready to be consumed by ansible-galaxy collection build

Right now, it requires a form of both, and that creates a very narrow and undocumented envelope where you can get this to work.

[mattclay]

@AlanCoding Source control is not required to use ansible-test. However, if git is being used, then only files seen by git will be seen by ansible-test. Copy the files with or without source control to a directory structure outside the current git checkout.

[AlanCoding]

Let me number these.

Workaround 1: abs path

Copy the files with or without source control to a directory structure outside the current git checkout.

I think what you're telling me looks like editing the target this way:

diff --git a/Makefile b/Makefile
index c0daed9048..42df8e5ccb 100644
--- a/Makefile
+++ b/Makefile
@@ -400,14 +400,13 @@ flake8_collection:
 test_collection_all: prepare_collection_venv test_collection flake8_collection test_collection_sanity
 
 test_collection_sanity:
-       rm -rf sanity
-       mkdir -p sanity
-       mkdir -p sanity/ansible_collections
-       mkdir -p sanity/ansible_collections/awx
-       cp -Ra awx_collection sanity/ansible_collections/awx/awx  # symlinks do not work
-       cd sanity/ansible_collections/awx/awx && ansible-playbook -i localhost, make_imports_absolute.yml  # hack because sanity tests do not fully work
-       cd sanity/ansible_collections/awx/awx && git init && git add . # requires both this file structure and a git repo, so there you go
-       cd sanity/ansible_collections/awx/awx && ansible-test sanity --test validate-modules
+       rm -rf /tmp/sanity
+       mkdir -p /tmp/sanity
+       mkdir -p /tmp/sanity/ansible_collections
+       mkdir -p /tmp/sanity/ansible_collections/awx
+       cp -Ra awx_collection /tmp/sanity/ansible_collections/awx/awx  # symlinks do not work
+       cd /tmp/sanity/ansible_collections/awx/awx && ansible-playbook -i localhost, make_imports_absolute.yml  # hack because sanity tests do not fully work
+       cd /tmp/sanity/ansible_collections/awx/awx && ansible-test sanity --test validate-modules
 
 build_collection:
        ansible-playbook -i localhost, awx_collection/template_galaxy.yml -e collection_package=$(COLLECTION_PACKAGE) -e namespace_name=$(COLLECTION_NAMESPACE) -e package_version=$(VERSION)

I ran and confirmed that works

Workaround 2: do not ignore

I confirmed that if I remove my testing folder from .gitignore, that does work:

diff --git a/.gitignore b/.gitignore
index 60d960b66e..be22f947ae 100644
--- a/.gitignore
+++ b/.gitignore
@@ -138,7 +138,6 @@ use_dev_supervisor.txt
 awx_collection_test_venv/
 awx_collection/*.tar.gz
 awx_collection/galaxy.yml
-sanity/
 
 .idea/*
 *.unison.tmp
diff --git a/Makefile b/Makefile
index c0daed9048..09f4594dbe 100644
--- a/Makefile
+++ b/Makefile
@@ -406,7 +406,6 @@ test_collection_sanity:
        mkdir -p sanity/ansible_collections/awx
        cp -Ra awx_collection sanity/ansible_collections/awx/awx  # symlinks do not work
        cd sanity/ansible_collections/awx/awx && ansible-playbook -i localhost, make_imports_absolute.yml  # hack because sanity tests do not fully work
-       cd sanity/ansible_collections/awx/awx && git init && git add . # requires both this file structure and a git repo, so there you go
        cd sanity/ansible_collections/awx/awx && ansible-test sanity --test validate-modules
 
 build_collection:

Workaround 3: git init hackery

This is what I have in #59991 (comment)

Exactly which workaround someone is going to like is a matter of preference, and I personally would rather do the git init hack to make subsequent manual testing easier. But I'm being very explicit about these because this is something that's really really hard for someone to track down.

[mattclay]

In the first example you should be able to simplify the directory creation to just the following since the -p option should create the intermediate directories:

mkdir -p /tmp/sanity/ansible_collections/awx

[bcoca]

closing as per above