Design a way to pass some configuration to integration tests

[marmarek]

Example things we may want to configure:

• which PCI device (on particular test machine) can be used for PCI passthrough tests
• path to the Windows installation image and/or installed disk image (for Windows tests)

We've already considered some command line options and it doesn't work with python unittest framework (which isn't surprising for unit test framework...). I can think of some environment variables, or separate config file.
cc @woju

[woju]

On Fri, Dec 04, 2015 at 01:39:48PM -0800, Marek Marczykowski-Górecki wrote:

Example things we may want to configure:

• which PCI device (on particular test machine) can be used for PCI passthrough tests
• path to the Windows installation image and/or installed disk image (for Windows tests)

We've already considered some command line options and it doesn't work
with python unittest framework (which isn't surprising for unit test
framework...). I can think of some environment variables, or separate
config file.

Config file would be hard, I think it's better to stick to
environment variables. Environment is in os.environ directory, which
can be queried like this:

import os
print(os.envion.get('QTB_CONFIG_OPTION', 'default value'))

which basically makes this a one-liner. If you'd like to skip the test
instead:

import os

class TC_00_MyTestCase(...):
    def test_000_test(self):
        try:
            conf = os.environ['QTB_CONFIG_OPTION']
        except KeyError:
            self.skip() # raises unittest.case.SkipTest

However, when many of those will pop up, I think it is better to get
those under one helper function, maybe in qubes.tests module:

import os
import unittest

def get_config_mandatory(key):
    try:
        return os.environ[key]
    except KeyError:
        raise unittest.SkipTest(
            'skipping because environment variable {!r} is unset'.format(key))

My .02 eurocents.

regards, .-.
Wojtek Porczyk .-^' '^-.
Invisible Things Lab |'-.-^-.-'|
| | | |
I do not fear computers, | '-.-' |
I fear lack of them. '-._ : ,-'
-- Isaac Asimov `^-^-_>

[woju]

On Fri, Dec 04, 2015 at 01:39:48PM -0800, Marek Marczykowski-Górecki wrote:

Example things we may want to configure:

• which PCI device (on particular test machine) can be used for PCI passthrough tests
• path to the Windows installation image and/or installed disk image (for Windows tests)

We've already considered some command line options and it doesn't work
with python unittest framework (which isn't surprising for unit test
framework...). I can think of some environment variables, or separate
config file.

Config file would be hard, I think it's better to stick to
environment variables. Environment is in os.environ directory, which
can be queried like this:

import os
print(os.envion.get('QTB_CONFIG_OPTION', 'default value'))

which basically makes this a one-liner. If you'd like to skip the test
instead:

import os

class TC_00_MyTestCase(...):
    def test_000_test(self):
        try:
            conf = os.environ['QTB_CONFIG_OPTION']
        except KeyError:
            self.skip() # raises unittest.case.SkipTest

However, when many of those will pop up, I think it is better to get
those under one helper function, maybe in qubes.tests module:

import os
import unittest

def get_config_mandatory(key):
    try:
        return os.environ[key]
    except KeyError:
        raise unittest.SkipTest(
            'skipping because environment variable {!r} is unset'.format(key))

My .02 eurocents.

regards, .-.
Wojtek Porczyk .-^' '^-.
Invisible Things Lab |'-.-^-.-'|
| | | |
I do not fear computers, | '-.-' |
I fear lack of them. '-._ : ,-'
-- Isaac Asimov `^-^-_>