Analyzer's execution order

[tiho44]

Hey Oleksii, small q from our side regarding artifacts:

we upload artifacts during fixture teardown. meaning testomatio_artifacts would not be ready on pytest_runtest_makereport call.when == 'call'

execution order, in our scenario:

1. pytest_runtest_makereport call=setup
2. pytest_runtest_makereport call=run
3. analyzer posts test report
4. fixture teardown (artifacts uploaded)

know you enabled s3_connector, but unfortunately it's not the way for us since fixtures have their own dependencies

this is the quick ugly patch that works for us:

def pytest_runtest_makereport(item: Item, call: CallInfo):
    global analyzer_config_option
    analyzer_config_option = item.config.getoption(analyzer_option)
    if analyzer_config_option is None or analyzer_config_option != 'sync':
        return
    elif not pytest.analyzer_test_run_config.test_run_id:
        return
    
    global request
    if call.when == 'setup':
        test_item = TestItem(item)
        request = {
            'status': None,
            'title': test_item.title,
            'run_time': None,
            'suite_title': test_item.file_name,
            'suite_id': None,
            'test_id': test_item.id[2:] if test_item.id else None,  # remove @T if exists
            'message': None,
            'stack': None,
            'example': None,
            'artifacts': None,
            'steps': None,
            'code': None,
        }
        if call.excinfo is not None:
            if call.excinfo.typename == 'Skipped':
                request['status'] = 'skipped'
            else:
                request['message'] = str(call.excinfo.value)
                request['stack'] = '\n'.join((str(tb) for tb in call.excinfo.traceback))
                request['status'] = 'failed'
        log.warning("setup req: %s", request)
    if call.when == 'call':
        request['run_time'] = call.duration
        if call.excinfo is not None:
            request['message'] = str(call.excinfo.value)
            request['stack'] = '\n'.join((str(tb) for tb in call.excinfo.traceback))
            request['status'] = 'failed'
        else:
            request['status'] = 'passed'
        if hasattr(item, 'callspec'):
            example = item.callspec.params
            if type(example) is bytes:
                request['example'] = example.decode('utf-8')
            elif type(example) in (str, int, float, bool):
                request['example'] = item.callspec.params
            else:
                request['example'] = 'object'  # to avoid json serialization error
        log.warning("call req: %s", request)
    if call.when == 'teardown':
        request['artifacts'] = getattr(item, 'testomatio_artifacts', None)
        log.warning("teardown req: %s", request)


def pytest_runtest_logfinish(nodeid, location):
    if analyzer_config_option is None or analyzer_config_option != 'sync':
        return
    elif not pytest.analyzer_test_run_config.test_run_id:
        return
     
    global request
    log.warning("Running logfinish: %s", request)
    if request['status']:
        log.warning("Uploading with status: %s", request['status'])
        connector = pytest.connector
        connector.update_test_status(run_id=pytest.analyzer_test_run_config.test_run_id, **request)
        for key,_ in request.items():
            request[key] = None

can this be done? i'm sure you have better ideas. thanks for your help.

[Ypurek]

sorry for the delay - just introduced changes you've requested