Add puppet tests for syncing and publishing with puppet_distributor

[peterlacko]

PR contains tests for following functionality:

• It is possible to create two puppet repos, with and without feed URLs respectively (CreateTestCase),
• Create repo with valid feed and run sync ends without errors for both valid and invalid query (SyncValidFeedTestCase),
• Create repo with invalid feed and run sync ends with error (SyncInvalidFeedTestCase),
• Following steps finish without errors (PublishTestCase)
  • Create two repositories: foo and bar.
  • upload custom puppet module to foo,
  • copy foo to bar and publish both,
  • query pulp server for modules using all query formats supported by given pulp version
  • download modules from their expected locations on pulp server file system,
  • verify integrity of downloaded modules.

[Ichimonji10]

Please rebase onto master.

[Ichimonji10]

Can we re-word this?

[peterlacko]

Thanks, done.

[Ichimonji10]

@placko, if the tests in this new module are in good shape, please run them and capture some test output, and put that test output in the commit message.

While we're at it, please update the commit message. I notice that the PR message here on GitHub is significantly more fleshed out than the commit itself.

[peterlacko]

@Ichimonji10 Ok, I will also split commits into two as is written above

[Ichimonji10]

Thank you.

[Ichimonji10]

Version handling logic has been added on the master branch. The big new difference is that pulp_smash.config.ServerConfig objects all now have a version attribute. This attribute is a packaging.version.Version object. These Version objects have quite intuitive usage:

>>> from packaging.version import Version
>>> Version('1') < Version('10')
True
>>> Version('2.7.3') > Version('2.7')
True
>>> Version('2.7.0') == Version('2.7')                                                                    
True

Version handling objects are useful for a number of reasons. Among others, they are superior to (codepoint-based) string comparisons:

>>> '1' < '10'  # OK
True
>>> '6' < '10'  # uh oh
False

They're also better than tuple comparisons in that they properly implement PEP 0440. While that PEP has come from the Python community, it's useful in the general sense, and compatible with semantic versioning. Should we want less strict version handling, we can also use packaging.version.parse. See more info here.

How do you use these new versioning objects in Pulp Smash? As mentioned, they're available on ServerConfig objects. Check it out:

>>> from pulp_smash.config import ServerConfig
>>> cfg = ServerConfig()
>>> type(cfg.version)
<class 'packaging.version.Version'>
>>> str(cfg.version)
'1!0'

What in the world is '1!0'? It's a valid version string - no, really! It reads "epoch 1, version 0". Any version with "epoch 1" or above will always be greater than a version in "epoch 0", and by default, every version is implicitly in epoch 0:

>>> from packaging.version import Version
>>> Version('1!0') > Version('2.6')                                                                       
True
>>> Version('1!0') > Version('3.1')                                                                       
True
>>> Version('0!3.1') == Version('3.1')
True

By the way, version string epochs aren't some wacky weird thing. They're widely recognized and used. I use them in some of the software packages I maintain, due to the fact that the developers have messed up and used incorrect version numbers, or due to developers changing numbering schemes.

Back to Pulp Smash.

How can we use these versions in our code? Let's take an example from this pull request. This bit of code is only valid on Pulp 2.7 and above:

        # Query modules on server. Data integrity is tested in tests below.
        query_href = '/v3/releases?module={}/{}'.format(
            _PUPPET_QUERY_PACKAGE['name'],
            _PUPPET_QUERY_PACKAGE['author']
        )
        cls.bodies['query units'] = tuple((
            _query_repo(
                cls.cfg,
                query_href,
                body['repo_id'],
                cls.responses['query units']
            ) for body in cls.bodies['distribute']
        ))

Now, I don't know the details of Pulp's behaviour here. But I'm going to assume that Pulp 2.6 and older also let you query for puppet modules, but with a different query. You can add in some conditional logic like so:

        # Query modules on server. Data integrity is tested in tests below.
        if cls.cfg.version < Version('2.7'):
            query_href = …
        else:
            query_href = '/v3/releases?module={}/{}'.format(
                _PUPPET_QUERY_PACKAGE['name'],
                _PUPPET_QUERY_PACKAGE['author']
            )
        cls.bodies['query units'] = tuple((
            _query_repo(
                cls.cfg,
                query_href,
                body['repo_id'],
                cls.responses['query units']
            ) for body in cls.bodies['distribute']
        ))

Just to be clear, this works as-is with absolutely no changes to the end user. An end user can now add a "version" attribute to their configuration file to explicitly state which version of Pulp they're testing against, like so:

{
    "default": {
        "version": "2.6.3",
        "base_url": "https://192.168.121.222",
        "verify": false,
        "auth": ["admin", "admin"]
    }
}

But this is entirely optional. Again, by default, Pulp Smash will assume that you're testing the most up-to-date development version of Pulp. It does this by automatically setting a vesrion of '1!0' if none is specified.

[dkliban]

We need to test that you can use the older versions of the puppet forge API (puppet clients prior to version 3.3).

On my dev environment I am able to run:

http --verify=no GET http://.:repo1@hostname/api/v1/releases.json?module=domcleal/katellovirt

In response I get json that tells me where that module can be downloaded. The basic auth in the above request is explained here [0].

We should also attempt to query the forge API used for clients between versions 3.3 and 3.6. The way the request is formulated is described here [1] These two additional scenarios will work on Pulp 2.6+, however the test that is already implemented will only work for Pulp version 2.7+.

[0] http://pulp-puppet.readthedocs.org/en/latest/tech-reference/forge_api.html#basic-auth
[1] http://pulp-puppet.readthedocs.org/en/latest/user-guide/recipes.html#installing-with-puppet-client-3-3

[Ichimonji10]

@dkliban I don't understand the bit about "we should attempt to query the forge API for clients between version 3.3 and 3.6.". Are you saying that the portion of the test case below this line is simply not applicable on Pulp 2.6 and older due to its use of an old pulp_python version or something? ELI5, please.

[peterlacko]

After discussion with @dkliban querying the module will look as follows:

• for pulp >= 2.7, 3 puppet queries formats will be tested, namely for puppet <=3.3, 3.3 - 3.5 and >=3.6
• for pulp <= 2.6 only queries from puppet <3.6 will be tested, as pulp in these versions doesn't support queries to new forge API [1]

I hope it makes sense.
[1] https://forgeapi.puppetlabs.com/

[Ichimonji10]

I think the index variable can be dropped:

        for call_report in cls.bodies['publish']:
            poll_spawned_tasks(cls.cfg, call_report)

[peterlacko]

cls.bodies['publish'] is tuple so we have to iterate over it like

for publish_report in cls.bodies['publish']:
    for call_report in publish_report:
        poll_spawned_tasks(cls.cfg, call_report)

but got the idea, avoid hard coded indexes whenever possible.

[Ichimonji10]

Thanks for the correction. Yes, the idea is to just avoid hard coded indices.

[Ichimonji10]

FYI, I've moved added functions to module pulp_smash.utils:

• get_importers
• publish_repository
• sync_repository

They should be usable in module pulp_smash.tests.puppet.api_v2.test_sync_publish.

[dkliban]

Import a puppet module from an upload request into a repository.

[peterlacko]

Thanks.

[Ichimonji10]

@peterlacko, I've walked through this pull request and given it a thorough once-over. Please take a look at master...Ichimonji10:puppet, and let me know if I've messed anything up.

[peterlacko]

@Ichimonji10: looks OK.

[Ichimonji10]

Merged as c0d8d08.