Test whether Pulp can create and publish new repo copied from another repo with older package

[danuzclaudes]

This is a follow-up issue after https://pulp.plan.io/issues/1684#change-10578 and Pulp-Smash Issue #190.

As @mhrivnak has pointed out thru his comments as the 5th note, a new test case will be setup as following:

1. create repo1 and sync from upstream.
2. find an RPM that has two different versions in the repo.
3. create a new repo2, and copy the older of the rpm version into it.
4. setup that repo2 on a host, and install the packages.
5. copy a newer version into the repo2 and re-publish
6. yum update on the client, and see what happens.

[Ichimonji10]

But this will be left to future work since currently pulp-smash does not have utility to setup a client vm.

Pulp Smash will never, ever have the ability to set up a VM.

This test can already be implemented in Pulp Smash, including installing a package on a client.

[Ichimonji10]

See: master...Ichimonji10:test-agent-demo

[Ichimonji10]

Can you update the name of this issue, please? When browsing through the list of issues, it'd be nice to see something like "Test whether Pulp can do foo," instead of "this issue is a follow up to another issue."

[danuzclaudes]

@Ichimonji10 one follow-up question with the pulp-agent.

I think this test will refer two servers only: upstream -> server1 (with repo1 + repo2) -> server2, where the server2 is supposed to install packages from server1 thru yum update. I'm not sure whether this step could be implemented thru pulp agent/client, since the pulp-smash only focuses on server for time being, right?

[Ichimonji10]

AFAIK, Pulp doesn't have any way of issuing commands to clients. My assumption is that the Pulp Smash system would execute commands on server2 via SSH, using the pulp_smash.cli.Client class.

pulp-smash only focuses on server for time being, right?

Yes, but that's purely coincidental. Pulp Smash can talk to other systems, and the only reason it hasn't happened yet is that nobody's made it happen.

[elyezer]

AFAIK, so far we have tests that involves Pulp components either on the same machine or in multiple (even though the later is not running at the moment).

We also need ability to have machines consuming Pulp content which we can call as being clients and so far there is no machinery in place for that. We can take advantage of the Pulp Smash settings and maybe expect some client- entries which would create a static environment or, on the other hand, we can have a entry for a client provisioning system and create VMs there which can build a dynamic environment and we can expect a clean client every time.

The first approach the responsibility to provide some clients is on the person which is running Smash and on the second approach the responsibility is to Smash create its own VMs, but the person which is running need to provide the system to get the machines provisioned.

[Ichimonji10]

AFAIK, Pulp doesn't have any way of issuing commands to clients.

I think I'm wrong about this. From the Pulp Installation docs:

Pulp operates with three main components that get installed potentially on different machines.

[…]

Agent
This component runs on consumers and communicates with the server to provide remote content management.

[…]

[preethit]

@Ichimonji10 I am a little confused about the issue.

Pulp consumer can

1. Install/Uninstall contents (rpm, errata etc) on a client

And the initial test described is doable. Let me try to write it up.

[Ichimonji10]

We also need ability to have machines consuming Pulp content which we can call as being clients and so far there is no machinery in place for that. We can…

In my opinion, we should keep VM-management logic out of Pulp Smash.

Right now, Pulp Smash has a pretty well defined interface. Users can tell Pulp Smash what system(s) are available via (by default) a ~/.config/pulp_smash/settings.json file, and Pulp Smash can choose which tests to run based on what's available. Easy enough. This well defined interface has at least the following benefits:

• Pulp Smash doesn't care about the nature of target systems: they can be VMs backed by libvirt, virtualbox, vmware, xen, etc, or docker containers, or bare-metal systems, or the local system, or even a mixture of these. This imparts a great deal of flexibility. Have you managed to get Pulp running on your gasoline-powered cheese grater? Great! Pulp Smash can test that system.
• Pulp Smash doesn't care where it's running from. Can you get the dependencies installed and provide password-less shell access to the target systems? Great! Pulp Smash can run on the current system.
• This well defined interface prevents scope creep. Pulp Smash is already able to talk to multiple systems via their API and CLI, and it does things like stop and start system services, drop databases, delete files from their filesystems, and so on. This is a reasonable level of complexity. Including VM management logic would necessarily add more complexity to Pulp Smash.

Now, I totally agree that it'd be nice to have multi-machine testing, where the environment is programmatically managed. So - how can we do that and still integrate nicely with Pulp Smash?

Well - use the settings.json interface! It's plain old JSON, and any programming language that speaks JSON can munge that file. Even better, if the tool that's managing VMs is Python (or has Python bindings?) (or can run python -c '...' commands?), have that tool use pulp_smash.config.ServerConfig to manage the settings.json file.

[preethit]

Expanding the initial test case.

Note: The pulp-client can be the server itself.

On a pulp-server

1. create repo1 and sync from upstream which has 2 versions of the same rpm

pulp-admin rpm repo create --repo-id repo1 --feed https://repos.fedorapeople.org/repos/pulp/pulp/demo_repos/test_multiple_rpm_versions/

pulp-admin rpm repo sync run --repo-id repo1

1. Find an RPM that has two different versions in the repo.

pulp-admin rpm repo content rpm --repo-id repo1 --str-eq "filename=grinder_test_package-11.0-1.fc14.noarch.rpm"

1. Create a new repo2, and copy the older of the rpm version into it.

pulp-admin rpm repo create --repo-id repo2
 pulp-admin rpm repo copy rpm -f repo1 -t repo2  --str-eq "filename=grinder_test_package-11.0-1.fc14.noarch.rpm"

pulp-admin rpm repo publish run --repo-id repo2

1. setup that repo2 on a host, and install the packages.
   This can be done 2 ways easily

Option 1

Setup pulp.repo  in /etc/yum.repos/ with the repo2 from above
Yum install the package

Option2

1. Install pulp-consumer packages
2. Register consumer to the pulp-server
3. Bind repo2 to the client
4. Use pulp-admin to install the package on the client
5. Copy a newer version into the repo2 and re-publish.

 pulp-admin rpm repo copy rpm -f repo1 -t repo2  --str-eq "filename=grinder_test_package-12.0-1.fc14.noarch.rpm"
 pulp-admin rpm repo publish run --repo-id repo2

1. yum update on the client, and see what happens. Implement this feature based on the comments below.

[danuzclaudes]

Hi folks, thank you all for your notes on this issue. I have updated the commit announcement fyi.

Btw, the test case that I have designed will install the RPM packages on the same pulp server, which creates repos, copies RPMs and then publishes the repository to itself. In that case, as a workaround, I don't need extra vms or installing consumers; while it can still verify the publishing. Please let me know if this design would work as issue automation. Thanks.

[elyezer]

Right now, Pulp Smash has a pretty well defined interface. Users can tell Pulp Smash what system(s) are available via (by default) a ~/.config/pulp_smash/settings.json file, and Pulp Smash can choose which tests to run based on what's available. Easy enough.

Agreed. It is better to leave the vm management outside the test framework. That will also match to the ssh access management which will be external as well, for example:

A tool can create the VM, update the ssh config and Pulp Smash settings and finally start the tests. We need just to define what will be the entry names expected to be the clients (cleanup will be needed because more than one test may need a client and if there is at least one they will run).

[elyezer]

PR is merged. This issue can be closed.