Skip to content

Commit

Permalink
Feature/drop python27 (#132)
Browse files Browse the repository at this point in the history
* removing python2.7 support

* using `if` `raise` over `assert`

* using `format` strings

* using `raise` instead of `exit`
  • Loading branch information
nklapste committed Jul 17, 2019
1 parent 1898081 commit 1d39a91
Show file tree
Hide file tree
Showing 10 changed files with 105 additions and 190 deletions.
9 changes: 2 additions & 7 deletions .travis.yml
Expand Up @@ -8,8 +8,6 @@ stages:

before_install:
- pip install codecov


install:
- pip install .
script:
Expand All @@ -28,9 +26,7 @@ jobs:
script:
- flake8 mutmut
- stage: test
python: "2.7"
- python: pypy
- python: "3.6"
python: "3.6"
- python: "3.7"
dist: xenial
- os: windows
Expand All @@ -44,8 +40,7 @@ jobs:
after_success:
- python -m codecov
- stage: deploy
python:
- 3.6
python: "3.6"
before_script: skip
after_script: skip
script:
Expand Down
84 changes: 53 additions & 31 deletions README.rst
Expand Up @@ -11,13 +11,16 @@ mutmut - python mutation tester
.. image:: https://codecov.io/gh/boxed/mutmut/branch/master/graph/badge.svg
:target: https://codecov.io/gh/boxed/mutmut

Mutmut is a mutation testing system for Python 2 and 3, with a strong focus on
ease of use. If you don't know what mutation testing is try starting with `this article <https://hackernoon.com/mutmut-a-python-mutation-testing-system-9b9639356c78>`_.
Mutmut is a mutation testing system for Python 3, with a strong focus on ease
of use. If you don't know what mutation testing is try starting with
`this article <https://hackernoon.com/mutmut-a-python-mutation-testing-system-9b9639356c78>`_.

Some highlight features:

- Found mutants can be applied on disk with a simple command making it very easy to work with the results
- Supports all test runners (because mutmut only needs an exit code from the test command)
- Found mutants can be applied on disk with a simple command making it very
easy to work with the results
- Supports all test runners (because mutmut only needs an exit code from the
test command)
- Extremely small and simple implementation (less than a thousand lines)
- Battle tested on tri.struct, tri.declarative, tri.form and tri.table
- Can use coverage data to only do mutation testing on covered lines
Expand All @@ -28,21 +31,22 @@ Install and run

You can get started with a simple:

.. code-block:: shell
.. code-block:: console
> pip install mutmut
> mutmut run
pip install mutmut
mutmut run
This will by default run pytest on tests in the "tests" or "test" folder and it will try to figure out where the code to mutate lies. Run
This will by default run pytest on tests in the "tests" or "test" folder and
it will try to figure out where the code to mutate lies. Run

.. code-block:: shell
.. code-block:: console
mutmut --help
for the available flags, to use other runners, etc. The recommended way to use mutmut if
the defaults aren't working for you is to add a block in `setup.cfg`. Then when you
come back to mutmut weeks later you don't have to figure out the flags again, just run
`mutmut run` and it works. Like this:
for the available flags, to use other runners, etc. The recommended way to use
mutmut if the defaults aren't working for you is to add a block in `setup.cfg`.
Then when you come back to mutmut weeks later you don't have to figure out the
flags again, just run `mutmut run` and it works. Like this:

.. code-block:: ini
Expand All @@ -53,17 +57,20 @@ come back to mutmut weeks later you don't have to figure out the flags again, ju
tests_dir=tests/
dict_synonyms=Struct, NamedStruct
You can stop the mutation run at any time and mutmut will restart where you left off. It's
also smart enough to retest only the surviving mutants when the test suite changes.
You can stop the mutation run at any time and mutmut will restart where you
left off. It's also smart enough to retest only the surviving mutants when the
test suite changes.

To print the results run `mutmut results`. It will give you output in the form of the commands to apply a mutation:
To print the results run `mutmut results`. It will give you output in the form
of the commands to apply a mutation:

.. code-block:: shell
.. code-block:: console
mutmut apply 3
You can just copy paste those lines and run and you'll get the mutant on disk. You should
REALLY have the file you mutate under source code control and committed before you mutate it!
You can just copy paste those lines and run and you'll get the mutant on disk.
You should **REALLY** have the file you mutate under source code control and
committed before you mutate it!


Whitelisting
Expand All @@ -75,54 +82,69 @@ You can mark lines like this:
some_code_here() # pragma: no mutate
to stop mutation on those lines. Some cases we've found where you need to whitelist lines are:
to stop mutation on those lines. Some cases we've found where you need to
whitelist lines are:

- The version string on your library. You really shouldn't have a test for this :P
- Optimizing break instead of continue. The code runs fine when mutating break to continue, but it's slower.
- Optimizing break instead of continue. The code runs fine when mutating break
to continue, but it's slower.


Example mutations
-----------------

- Integer literals are changed by adding 1. So 0 becomes 1, 5 becomes 6, etc.
- < is changed to <=
- `<` is changed to `<=`
- break is changed to continue and vice versa

In general the idea is that the mutations should be as subtle as possible. See `__init__.py` for the full list.
In general the idea is that the mutations should be as subtle as possible.
See `__init__.py` for the full list.


Workflow
--------

This section describes how to work with mutmut to enhance your test suite.

1. Run mutmut with `mutmut run`. A full run is preferred but if you're just getting started you can exit in the middle and start working with what you have found so far.
1. Run mutmut with `mutmut run`. A full run is preferred but if you're just
getting started you can exit in the middle and start working with what you
have found so far.
2. Show the mutants with `mutmut results`
3. Apply a surviving mutant to disk running `mutmut apply 3` (replace 3 with the relevant mutant ID from `mutmut results`)
3. Apply a surviving mutant to disk running `mutmut apply 3` (replace 3 with
the relevant mutant ID from `mutmut results`)
4. Write a new test that fails
5. Revert the mutant on disk
6. Rerun the new test to see that it now passes
7. Go back to point 2.

Mutmut keeps a result cache in `.mutmut-cache` so if you want to make sure you run a full mutmut run just delete this file.
Mutmut keeps a result cache in `.mutmut-cache` so if you want to make sure you
run a full mutmut run just delete this file.

You can also tell mutmut to just check a single mutant:

.. code-block:: shell
.. code-block:: console
> mutmut run 3
mutmut run 3
JUnit XML support
-----------------

In order to better integrate with CI/CD systems, `mutmut` supports the generation of a JUnit XML report (using https://pypi.org/project/junit-xml/).
This option is available by calling `mutmut junitxml`. In order to define how to deal with suspicious and untested mutants, you can use `mutmut junitxml --suspicious-policy=ignore --untested-policy=ignore`.
In order to better integrate with CI/CD systems, `mutmut` supports the
generation of a JUnit XML report (using https://pypi.org/project/junit-xml/).
This option is available by calling `mutmut junitxml`. In order to define how
to deal with suspicious and untested mutants, you can use

.. code-block:: console
mutmut junitxml --suspicious-policy=ignore --untested-policy=ignore
The possible values for these policies are:

- `ignore`: Do not include the results on the report at all
- `skipped`: Include the mutant on the report as "skipped"
- `error`: Include the mutant on the report as "error"
- `failure`: Include the mutant on the report as "failure"

If a failed mutant is included in the report, then the unified diff of the mutant will also be included for debugging purposes.
If a failed mutant is included in the report, then the unified diff of the
mutant will also be included for debugging purposes.
20 changes: 6 additions & 14 deletions mutmut/__init__.py
@@ -1,9 +1,6 @@
# -*- coding: utf-8 -*-

from __future__ import unicode_literals

import re
import sys

from parso import parse
from parso.python.tree import Name, Number, Keyword
Expand All @@ -18,7 +15,7 @@ def __init__(self, line, index, line_number):
self.line_number = line_number

def __repr__(self):
return 'MutationID(line="%s", index=%s, line_number=%s)' % (self.line, self.index, self.line_number)
return 'MutationID(line="{}", index={}, line_number={})'.format(self.line, self.index, self.line_number)

def __eq__(self, other):
return (self.line, self.index, self.line_number) == (other.line, other.index, other.line_number)
Expand Down Expand Up @@ -85,11 +82,13 @@ def matches(self, node, pattern=None, skip_child=None):
check_value = True
check_children = True

# Match type based on the name, so _keyword matches all keywords. Special case for _all that matches everything
# Match type based on the name, so _keyword matches all keywords.
# Special case for _all that matches everything
if pattern.type == 'name' and pattern.value.startswith('_') and pattern.value[1:] in ('any', node.type):
check_value = False

# The advanced case where we've explicitly marked up a node with the accepted types
# The advanced case where we've explicitly marked up a node with
# the accepted types
elif id(pattern) in self.marker_type_by_id:
if self.marker_type_by_id[id(pattern)] in (pattern.type, 'any'):
check_value = False
Expand Down Expand Up @@ -140,13 +139,6 @@ def matches(self, node, pattern=None, skip_child=None):
]


if sys.version_info < (3, 0): # pragma: no cover (python 2 specific)
# noinspection PyUnresolvedReferences
text_types = (str, unicode) # noqa: F821
else:
text_types = (str,)


UNTESTED = 'untested'
OK_KILLED = 'ok_killed'
OK_SUSPICIOUS = 'ok_suspicious'
Expand Down Expand Up @@ -475,7 +467,7 @@ def mutate(context):
try:
result = parse(context.source, error_recovery=False)
except Exception:
print('Failed to parse %s. Internal error from parso follows.' % context.filename)
print('Failed to parse {}. Internal error from parso follows.'.format(context.filename))
print('----------------------------------')
raise
mutate_list_of_nodes(result, context=context)
Expand Down
42 changes: 8 additions & 34 deletions mutmut/__main__.py
@@ -1,15 +1,14 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import print_function

import fnmatch
import itertools
import os
import shlex
import subprocess
import sys
import traceback
from configparser import ConfigParser, NoOptionError, NoSectionError
from functools import wraps
from io import open
from os.path import isdir, exists
Expand All @@ -29,31 +28,6 @@

spinner = itertools.cycle('⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏')

if sys.version_info < (3, 0): # pragma: no cover (python 2 specific)
# noinspection PyCompatibility,PyUnresolvedReferences
from ConfigParser import ConfigParser, NoOptionError, NoSectionError
# This little hack is needed to get the click tester working on python 2.7
orig_print = print

def print(x='', **kwargs):
# noinspection PyUnresolvedReferences
x = x.decode("utf-8")
orig_print(x.encode("utf-8"), **kwargs)

# noinspection PyShadowingBuiltins
class TimeoutError(OSError):
"""Defining TimeoutError for Python 2 compatibility"""

# noinspection PyShadowingBuiltins
class FileNotFoundError(OSError):
"""Defining FileNotFoundError for Python 2 compatibility"""
else:
# noinspection PyUnresolvedReferences,PyCompatibility
from configparser import ConfigParser, NoOptionError, NoSectionError

# noinspection PyShadowingBuiltins
TimeoutError = TimeoutError


# decorator
def config_from_setup_cfg(**defaults):
Expand Down Expand Up @@ -190,7 +164,7 @@ def __init__(self, swallow_output, test_command, exclude_callback,
self.pre_mutation = pre_mutation

def print_progress(self):
print_status('%s/%s 🎉 %s%s 🤔 %s 🙁 %s' % (self.progress, self.total, self.killed_mutants, self.surviving_mutants_timeout, self.suspicious_mutants, self.surviving_mutants))
print_status('{}/{} 🎉 {}{} 🤔 {} 🙁 {}'.format(self.progress, self.total, self.killed_mutants, self.surviving_mutants_timeout, self.suspicious_mutants, self.surviving_mutants))


DEFAULT_TESTS_DIR = 'tests/:test/'
Expand Down Expand Up @@ -265,18 +239,18 @@ def main(command, argument, argument2, paths_to_mutate, backup, runner, tests_di
:rtype: int
"""
if version:
print("mutmut version %s" % __version__)
print("mutmut version {}".format(__version__))
return 0

if use_coverage and use_patch_file:
raise click.BadArgumentUsage("You can't combine --use-coverage and --use-patch")

valid_commands = ['run', 'results', 'apply', 'show', 'junitxml']
if command not in valid_commands:
raise click.BadArgumentUsage('%s is not a valid command, must be one of %s' % (command, ', '.join(valid_commands)))
raise click.BadArgumentUsage('{} is not a valid command, must be one of {}'.format(command, ', '.join(valid_commands)))

if command == 'results' and argument:
raise click.BadArgumentUsage('The %s command takes no arguments' % command)
raise click.BadArgumentUsage('The {} command takes no arguments'.format(command))

dict_synonyms = [x.strip() for x in dict_synonyms.split(',')]

Expand Down Expand Up @@ -387,7 +361,7 @@ def _exclude(context):
return False

if command != 'run':
raise click.BadArgumentUsage("Invalid command %s" % command)
raise click.BadArgumentUsage("Invalid command {}".format(command))

mutations_by_file = {}

Expand Down Expand Up @@ -694,7 +668,7 @@ def feedback(line):
if returncode == 0 or (using_testmon and returncode == 5):
baseline_time_elapsed = time() - start_time
else:
raise RuntimeError("Tests don't run cleanly without mutations. Test command was: %s\n\nOutput:\n\n%s" % (test_command, '\n'.join(output)))
raise RuntimeError("Tests don't run cleanly without mutations. Test command was: {}\n\nOutput:\n\n{}".format(test_command, '\n'.join(output)))

print(' Done')

Expand Down Expand Up @@ -723,7 +697,7 @@ def add_mutations_by_file(mutations_by_file, filename, exclude, dict_synonyms):
mutations_by_file[filename] = list_mutations(context)
register_mutants(mutations_by_file)
except Exception as e:
raise RuntimeError('Failed while creating mutations for %s, for line "%s"' % (context.filename, context.current_source_line), e)
raise RuntimeError('Failed while creating mutations for {}, for line "{}"'.format(context.filename, context.current_source_line), e)


def python_source_files(path, tests_dirs, paths_to_exclude=None):
Expand Down

0 comments on commit 1d39a91

Please sign in to comment.