does pytest support to this way to collect test cases?

[hanks]

I want to shorten test method name, and just rewrite method name with comment, like this:

def _test(comment):
    def middle(func):
        def wrapper(*args, **kwargs):
            func.__name__ = 'test_' + comment
            func.__doc__ = comment
            return func(*args, **kwargs)
        return wrapper
    return middle


class TestCase(object):
    @ _test('true')
    def _(self):   # this method name will be test_true
        assert 0 == 0

    @ _test('false')
    def _(self):   # this method name will be test_false
        assert 1 == 0

Yeah, it does not work, but how to customise pytest to implement this style?

[RonnyPfannschmidt]

Instead if making a wrapper functioning, just add a __test__ attribute with a true value

Also you can override collection from the pytest side

A native pytest.mark.test is in planning

But can also easily be implemented in a conftest

[RonnyPfannschmidt]

the most simple method should be like (i didn't run it on my machine and it needs a recent py.test)

def test(func):
  func.__test__ = True
  return func


class TestCase(object):
    @test
    def true(self):   # this method name will be test_true
        assert 0 == 0

    @test
    def false(self):   # this method name will be test_false
        assert 1 == 0

also closing the issue for now

[hanks]

@RonnyPfannschmidt Thank you for your comment. But that seems to be not the style I want.
I want to use the same name for each method for test, and use comment to build test method name dynamical, like:

def _test(comment):  # can not use `test`, because it will be collected if do not change the default 
    def middle(func):
        def wrapper(*args, **kwargs):
            func.__test__ = True
            func.__name__ =  '_'.join(['test'] + comment.split(' '))  # build test method name with comment
            func.__doc__ = comment
            return func(*args, **kwargs)
        return wrapper
    return middle


class TestCase(object):
    @ _test('true condition when 0')
    def _(self):   # this method name will be `test_true_condition_when_0`  dynamically
        assert 0 == 0

    @ _test('false condition when 1')
    def _(self):   # this method name will be `test_false_confition_when_1` dynamically
        assert 1 == 0

python does not support method overload, and I tried, but the result will be just only 1 test method collected, how to custom pytest to collect like this way?

[RonnyPfannschmidt]

you can use call stack inspection, and put the tests into the locals of the calling namespace

or you create a custom object in the locals to begin with

class Testcase(object):
   _test = TestStore()
   @_test('hi there')
   def _(self):
     pass

you will need to extend the reporting as well if you want to use this style

[iamveritas]

is there anywhere documented how to migrate "nose2 type of tests that yield self.assert* and each yield is collected as a separate test" into pytest?
I;ve been searching with no good result an easy way to migrate a quite large of nose2 tests into py.test, so any help would be appreciated.
I somehow see the question raised here related with my problem cause here also it is needed to generate tests dynamically which will be discovered and recognised by pytest
thank you

[RonnyPfannschmidt]

@Oviwankenobi that kind of test got deprecated in pytest as its not compatible with setupstate and fixtures

currently its sugested not to do them,
a correct way to express such checks is planned but needs volunteers

[nicoddemus]

I;ve been searching with no good result an easy way to migrate a quite large of nose2 tests into py.test, so any help would be appreciated.

Automatically no, our suggestion is to convert them to parametrized tests: https://docs.pytest.org/en/latest/parametrize.html?highlight=parametrize. It is usually straightforward, but if you get into trouble you can post the test code and we can help converting it.

[RonnyPfannschmidt]

@nicoddemus i presume @Oviwankenobi is facing the same issue moinmoin faced - pytest completely messed up context-bearing yield tests - and it wont ever redeem them

[iamveritas]

@nicoddemus @RonnyPfannschmidt thank you for your input

I am not sure how parameterize feature can help but maybe I am not getting it right so here's the tests pattern I am looking at

def test_case_1
   for each input_test_file in list_of_files
      yield from _check_file(input_test_file)

def _check_file(input_file)
   yield true/false

list_of_files is dynamic, no hardcoded option here.

[nicoddemus]

@Oviwankenobi is the list of files obtained at collection time, IOW roughly during importing of the test module? If it is:

@pytest.mark.parametrize('input_test_file', list_of_files)
def test_case_1(input_test_file):
    assert _check_file(input_test_file)

However if list_of_files can only be realized after the collection and right before the test, then unfortunately there's not a direct mechanism to support that in Pytest I'm afraid.

[RonnyPfannschmidt]

@Oviwankenobi take a look at sybil perhpas - its about generating test items from parsed files and supports supplying own parsers * has a basic pytest integration

[iamveritas]

thanks guys, I end up developing a custom solution: a base class for all our tests which implements an asserts method which knows to collect all errors instead of stopping the test and at the end add to the final test report the collected errors.
the problem I have now is that the pytest --verbose --junitxml is not including those errors I have collected and added to the final report myself; any idea what am I missing?
those collected errors which I add myself show up in the output of the test execution but not in the junitxml file.

[nicoddemus]

Hmm hard to tell without looking at the code... does this base class subclass unittest.TestCase? If not, how do you "capture" the errors?

[iamveritas]

in pytest you do not have to derive your test from a base class of unittest.TestCase

the errors I collect are those of type "assert errors (e.g. val1 == val2)", I collect them in an dictionary and through the below hook I add them into the final report

@pytest.hookimpl(hookwrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    # do whatever you want before the next hook executes
    outcome = yield

    # outcome.excinfo may be None or a (cls, val, tb) tuple
    res = outcome.get_result()  # will raise if outcome was exception

    if res.when == 'call':
        if not internal_errors == 0:
           #add all collected errors 
           for error in test_results[caller_function_name][False]:
              res.sections.append(("Captured stder call", "ERROR: (%s): %s" % (caller_function_name, error)))
           res.sections.append(("Captured stdout call", "Test case final result FAILED"))

[iamveritas]

I am not sure if I am more clear in this post ... :) thanks a lot for your time
https://stackoverflow.com/questions/44406150/pytest-how-to-change-result-of-the-test