pytest support

[hroncok]

Just found out about this from @ana-balica and I love it. However from the README, I'm unsure whether mutpy works with unittest only, or if it also works with pytest. Could you please clarify? Also, what would be needed to make ti work, if it currently doesn't?

[konradhalas]

Hi @hroncok! Thank you for your interest :) MutPy supports only unittest currently, but pytest support is on our roadmap with very high priority.

What would be needed to make it work? We have to rewrite interaction with tests in MutationController and probably extract it to something more general. PR's and ideas are welcome!

@phihos do you have any thoughts in this topic?

[hroncok]

PR's and ideas are welcome!

Honestly I have no idea how this works. I'll be happy to help with testing.

Do you think pytest support should be a first class citizen or pytest-mutpy package / plugin should be created instead?

[phihos]

I already looked a bit into this: As Konrad mentioned, the code running the tests lives inside the MutationController. I also agree, that it needs to be extracted to a separate class before thinking about implementing pytest.

I suggest the following approach:

1. Extract test running code to a separate class.
2. Seprate this class into BaseTestRunner and UnittestRunner (child of BaseTestRunner)
3. Implement PyTestRunner as child of BaseTestRunner.
4. Offer possibility to choose the test runner class as command line parameter.

Do you think pytest support should be a first class citizen or pytest-mutpy package / plugin should be created instead?

I already thought about this, too. I am still not sure what approach to take on this matter.

Edit: After some thinking I came to the conclusion that separating different test runners into extra packages makes more sense.
If we would keep them in the main package, we have to

1. either force users to install all runners (pytest/nosetest etc)
2. or omit these test runner dependencies, so that the user has to manually install the needed test runner.

Possibility 1 is obvously not optimal since the user has to install unnecessary dependencies. Possibility 2 is not good either, because we would have no control over the test runner versions in use. This makes it harder to deprecate older versions of pytest/nosetest.
Creating packages like mutpy-pytest and mutpy-nose allows us to deprecate test runner versions and the user installs only necessary dependencies.
Only unittest should be already included, since it requires no extra dependencies.

[boxed]

@hroncok you might want to check out mutmut, my mutation tester. It supports all test runners because it only requires a process exit code to communicate with runners. The Python 3 support is limited by the lib I use for mutation though, so there are some limitations there.

Mutmut also has some significant upsides over mutpy, the most important being that you can apply mutations in disk to your code and it doesn’t reformat the entire file and throw away comments. Mutmut also has a super tiny and simple code base so it’s easy to modify if you find you need some mutation that is missing.

[sobolevn]

I would be happy to see pytest support. Sadly, I don't have the time to contribute.

[boxed]

@sobolevn @hroncok mutmut now supports python 3 and 2 fully. I've also made some big improvements to the usability and documentation. Check it out!

[phihos]

@boxed Please refrain from using our issues as advertising platform. You would not want us to pollute your issues either, would you?

@sobolevn I currently have more free time and I will continue to work on this feature. In the meantime you can check out the branch externalize-test-runner. In this branch some experimental pytest support already exists.
After downloading you go into the extracted directory and install it with pip install ..
You must run MutPy with the --runner pytest option to enable pytest support.

[Xophe92]

Hi all,

Thank you very much for this very interesting tool !

@phihos the --runner pytest flag you mentioned above seem now to be deployed as I had it after pip installing mutpy.

I though experiment issues that I have not in unittest ; I made up a repo to showcase this.
All the mutants of a pip install -e . installed package survive in pytest but are killed with unittest.

demo of my issue

I would like to know the status of this issue, if there are workarounds and if I could be of any help.

once again, great job 👍