ansible-test does not work with collections installed by galaxy inside a git repository

[felixfontein]

SUMMARY

For example in ansible.netcommon installed with ansible-galaxy collection install ansible.netcommon:

$ ansible-test integration -vvv
RLIMIT_NOFILE: (1024, 524288)
MAXFD: -1
ERROR: Cannot run integration tests without "roles/test/" or "tests/integration/targets/".

$ ansible-test units -vvv
RLIMIT_NOFILE: (1024, 524288)
MAXFD: -1
ERROR: Cannot run unit tests without "tests/unit/".

$ ansible-test sanity -vvv
RLIMIT_NOFILE: (1024, 524288)
MAXFD: -1
WARNING: All targets skipped.

That collection has both unit and integration tests, and some plugins and modules.

For source collections checked out with git, ansible-test works.

ISSUE TYPE

• Bug Report

COMPONENT NAME

ansible-test

ANSIBLE VERSION

devel

[ansibot]

Files identified in the description:

• bin/ansible-test

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[DouglasHeriot]

I'm currently dealing with this issue as well. Not sure if it's related to #63032 - is ansible_collections inside a git repo higher up, with ansible_collections in .gitignore?

[felixfontein]

I think @sivel mentioned on IRC that he cannot reproduce it. I still get this, though. And I don't have ansible_collections in .gitignore, so it's not related to that (maybe you also have that problem, but then that's a different problem :) ). Let me tinker with ansible-test a bit, maybe I can get an idea.

[felixfontein]

Ok, I think I see what the problem is. I installed the collection in a place which is part of a git repository higher up. ansible_collections is not in .gitignore, but a directory a few levels higher up is. So yeah, this is exactly #63032. For me, ~/projecs is a git repository, which somewhere in it contains github-cloned (which is in .gitignnore), and that eventually contains ansible_collections.

The problem is that ansible-test first tries to find a Git repository containing the collection, and only if that fails uses the UnversionedSource provider: https://github.com/ansible/ansible/blob/devel/test/lib/ansible_test/_internal/data.py#L123-L126

I was able to fix the problem as follows:

diff --git a/test/lib/ansible_test/_internal/data.py b/test/lib/ansible_test/_internal/data.py
index 6f8cbf45ca..fefe8fcd9c 100644
--- a/test/lib/ansible_test/_internal/data.py
+++ b/test/lib/ansible_test/_internal/data.py
@@ -121,7 +121,7 @@ class DataContext:
         layout_provider = find_path_provider(LayoutProvider, layout_providers, root, walk)
 
         try:
-            source_provider = find_path_provider(SourceProvider, source_providers, root, walk)
+            source_provider = find_path_provider(SourceProvider, source_providers, root, walk, True)
         except ProviderNotFoundForPath:
             source_provider = UnversionedSource(layout_provider.root)
 
diff --git a/test/lib/ansible_test/_internal/provider/__init__.py b/test/lib/ansible_test/_internal/provider/__init__.py
index 6e034b536b..0404f91e17 100644
--- a/test/lib/ansible_test/_internal/provider/__init__.py
+++ b/test/lib/ansible_test/_internal/provider/__init__.py
@@ -29,6 +29,7 @@ def find_path_provider(provider_type,  # type: t.Type[TPathProvider],
                        provider_classes,  # type:  t.List[t.Type[TPathProvider]]
                        path,  # type: str
                        walk,  # type: bool
+                       check_paths=False,
                        ):  # type: (...) -> TPathProvider
     """Return the first found path provider of the given type for the given path."""
     sequences = sorted(set(pc.sequence for pc in provider_classes if pc.sequence > 0))
@@ -40,7 +41,12 @@ def find_path_provider(provider_type,  # type: t.Type[TPathProvider],
         while True:
             for provider_class in tier_classes:
                 if provider_class.is_content_root(candidate_path):
-                    return provider_class(candidate_path)
+                    provider = provider_class(candidate_path)
+                    if check_paths:
+                        if path in provider.get_paths(path):
+                            return provider
+                    else:
+                        return provider
 
             if not walk:
                 break

I.e. only select source providers which actually find the path to the collection.

@mattclay would a change like this be OK?

[felixfontein]

This patch breaks some other cases (like change detection), since path is never in provider.get_paths(path). I've adjusted it and I think I found a good solution.

resolved_by_pr #69341

[ebuildy]

Any workaround for this?

[l3ender]

It took me quite awhile to realize I'm facing the same issue. In my scenario, my home directory (~) is a git repo to version my config files. As a result, I cannot run ansible-test in the installed collection directory (~/.ansible/collections).

Is there a recommended workaround?

@felixfontein - Any reason why you closed the PR with the fix?

[felixfontein]

@l3ender the easiest workaround right now is to run git init . in the installed collection (or in any top level directory where it might make sense).

I closed the PR because @mattclay wanted to solve it in a different way.

[nitzmahone]

There's not an easy apparent solution to this one that we can think of, so instead of continuing to punt this as a "maybe someday", we're just going to close it. If someone comes up with a more robust approach to detect and account for the issue in ansible-test, feel free to reopen or propose a change.