Allow loading dirs from role defaults/vars

[agaffney]

SUMMARY

This commit moves code to look for vars files/dirs to a common place and uses it for loading role defaults/vars. This allows things such as defaults/main or vars/main being a directory in a role, allowing splitting of defaults/vars into multiple files. This commit also fixes the role loading unit tests for py3 when bytestrings are used for paths instead of utf8 strings.

This fixes #14248 and #11639.

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

vars loading

ANSIBLE VERSION

N/A

ADDITIONAL INFORMATION

N/A

[agaffney]

@bcoca

[agaffney]

Yes, that should probably be continue. I work in a lot of different languages and those kinds of things are easy to mix up. That code path must not be exercised much, since it didn't break any of the tests and nobody has screamed yet.

[agaffney]

I'm having a hard time getting ansible to even hit that line of code. It should only get there on a role subdir such as tasks/ that uses allow_dir=False and when a matching directory is found before a file, but I can't get it to fail when I add a unit test like that.

[andriilahuta]

I'm not familiar with ansible internals, but this seemed to do the trick:

    @patch('ansible.playbook.role.definition.unfrackpath', mock_unfrackpath_noop)
    def test_load_role_with_tasks_dir_vs_file(self):

        fake_loader = DictDataLoader({
            "/etc/ansible/roles/foo_tasks/tasks/custom_main/foo.yml": """
            - command: bar
            """,
            "/etc/ansible/roles/foo_tasks/tasks/custom_main.yml": """
            - command: baz
            """,
        })

        mock_play = MagicMock()
        mock_play.ROLE_CACHE = {}

        i = RoleInclude.load('foo_tasks', play=mock_play, loader=fake_loader)
        r = Role.load(i, play=mock_play, from_files=dict(tasks='custom_main'))

        self.assertEqual(r._task_blocks[0]._ds[0]['command'], 'baz')

[agaffney]

Yeah, that seems to trigger the bug (or something that could be attributed to it). I did something very similar, but I wasn't using tasks_from, which appears to be necessary to trigger it.

[luckylittle]

It doesn't really work as i would expect it to work (why does 3rd scenario not work?). I tested different scenarios and this is what i found:

1. Doesn't work:

defaults/file1.yml
defaults/file2.yml
defaults/main.yml

2. Doesn't work:

defaults/file1.yml
defaults/file2.yml
defaults/file3.yml

3. Doesn't work:

defaults/main/file1.yml
defaults/main/file2.yml
defaults/main.yml

4. Works:

defaults/main/file1.yml
defaults/main/file2.yml
defaults/main/main.yml

If this is intended, i am happy to update Playbooks Best Practices.