Define a new and more flexible configuration syntax

[vkarak]

The idea is to have everything under site_configuration, which itself should be formatted as a JSON object in Python. This will allow us to accept easily .json config files and later YAML. Additionally, all configuration sections must be allowed to be configured per system or partition (like the environments, but with a different syntax). Here is a tentative proposal for the syntax:

site_configuration = {
    'systems': [
        {
            'name': 'generic',
            'descr': 'Generic example system',
            'hostnames': ['localhost'],
            'partitions': [
                {
                    'name': 'login',
                    'scheduler': 'local',
                    'modules': [],
                    'access':  [],
                    'environs': ['builtin-gcc'],
                    'descr': 'Login nodes'
                }
            ]
        }
    ],

    'environments': [
        {
            'name': 'builtin',
            'target_systems': ['*'],
            'cc':  'cc',
            'cxx': '',
            'ftn': '',
        },
        {
            'name': 'builtin-gcc',
            'target_systems': ['*'],
            'cc':  'gcc',
            'cxx': 'g++',
            'ftn': 'gfortran',
        }
    ],

    'schedulers': [
        {
            'name': 'slurm',
            'target_systems': ['*'],
            'wait_poll_intervals': [1, 2, 3],
            'submit_timeout': 60
        }
    ],

    'logging': [
        {
            'target_systems': ['*'],
            'level': 'DEBUG',
            'handlers': [
                {
                    'type': 'file',
                    'name': 'reframe.log',
                    'level': 'DEBUG',
                    'format': '[%(asctime)s] %(levelname)s: '
                    '%(check_info)s: %(message)s',
                    'append': False,
                },

                # Output handling
                {
                    'type': 'stream',
                    'name': 'stdout',
                    'level': 'INFO',
                    'format': '%(message)s'
                },
                {
                    'type': 'file',
                    'name': 'reframe.out',
                    'level': 'INFO',
                    'format': '%(message)s',
                    'append': False,
                }
            ]
        }
    ],

    'perf_logging': [
        {
            'target_systems': ['*'],
            'level': 'DEBUG',
            'handlers': [
                {
                    'type': 'filelog',
                    'prefix': '%(check_system)s/%(check_partition)s',
                    'level': 'INFO',
                    'format': (
                        '%(asctime)s|reframe %(version)s|'
                        '%(check_info)s|jobid=%(check_jobid)s|'
                        '%(check_perf_var)s=%(check_perf_value)s|'
                        'ref=%(check_perf_ref)s '
                        '(l=%(check_perf_lower_thres)s, '
                        'u=%(check_perf_upper_thres)s)|'
                        '%(check_perf_unit)s'
                    ),
                    'append': True
                }
            ]
        }
    ],

    'general': [
        {
            'target_systems': ['*'],
            'check_path': ['checks/'],
            'check_path_recurse': True
        }
    ]
}

The target_systems will be optional and if not specified differently, it will default to ['*'].

This issue should take care of defining the JSON schema for the configuration and of providing a first implementation that will read and load the configuration.