Create mechanism for plugging tests from the outside of core-admin repository

[marmarek]

It is really convenient to have a single test runner to launch all the Qubes tests. But some of them are barely related to core-admin repository (for example split GPG). Additionally (especially for qubes-app-*) it would be useful to have the same test running on any Qubes version (supported by such component).

This means that we need some API to allow tests from other repositories to be plugged into test runner (qubes.tests.run module). As discussed in #1368 we've settled on using entry points for that and providing some minimal stable API for creating VMs for the test purposes.

The API

External test case needs to expose entry point(s). Currently supported two of them:

• qubes.tests.extra - test should be instantiated once
• qubes.tests.extra.for_template - test should be instantiated for every template (generally should be used to testing some VM software)

Each of those entry points should be a function, which will return a list of classes derived from unittest.TestCase. This class will be mixed with Qubes-version specific class providing basic API for handling VMs:

• create_vms(names) - create AppVMs with names given as a parameter (iterable of names); return list of created objects
• enable_network() - created VMs should have access to the network (if available on the host)
• qrexec_policy(service, source, destination) - allow qrexec calls described by parameters
• shutdown_and_wait(vm, timeout=60) - ask given VM (VM object) to shutdown and wait for it to finish
• remove_vms(vms) - remove VMs (list of VM objects)
• save_and_reload_db() - ensure the changes are written to the disk (create_vms and remove_vms call it automatically, but if you make any additional changes, you need to call it on your own)

Additionally some useful functions are available:

• wait_for_window(title, timeout=30, show=True) - wait for given window to show (or hide if show=False)
• enter_keys_in_window(title, keys) - enter given keys into given window

Test case should take care of any other API compatibility on its own. In case of not supported API detected, it should call self.skipTest with appropriate message. For example:

try:
    vm.some_core_dependent_function(or_, core, dependent, params)
except (AttributeError, TypeError): 
    self.skip('core API mismatch')

[marmarek]

Automated announcement from builder-github

The package qubes-core-dom0-3.1.14-1.fc20 has been pushed to the r3.1 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-dom0-3.1.14-1.fc20 has been pushed to the r3.1 testing repository for dom0.
To test this update, please install it with the following command:

sudo qubes-dom0-update --enablerepo=qubes-dom0-current-testing

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-dom0-3.1.14-1.fc20 has been pushed to the r3.1 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update

[marmarek]

Automated announcement from builder-github

The package qubes-core-dom0-3.1.14-1.fc20 has been pushed to the r3.1 stable repository for dom0.
To install this update, please use the standard update command:

sudo qubes-dom0-update

Or update dom0 via Qubes Manager.

Changes included in this update