Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,10 @@
0.9.9 (unreleased)
------------------

- Nothing changed yet.
- Reduce test run noise by blacklisting annoying stdlib objects.
Issue `#12 <https://github.com/nodev-io/pytest-nodev/issues/12>`_.
- Report object that pass a test as **PASSED**, phase out HIT concept altogether.
- Get docs and dependencies in shape for the 1.0.0 release.


0.9.8 (2016-03-16)
Expand Down
28 changes: 14 additions & 14 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ To be more precise pytest-nodev is a `pytest <https://pytest.org>`_ plugin
that lets you execute a set of tests that specify the expected behaviour of a class or a function
on all objects in the Python standard library and in all the modules you have installed.

**Show me how it works in practice.**
**I need to write a** ``parse_bool`` **function that robustly parses a boolean value from a string.**
**Here is the test I intend to use to validate my own implementation once I write it.**::

Expand All @@ -34,8 +35,6 @@ on all objects in the Python standard library and in all the modules you have in
assert parse_bool('TRUE')
assert parse_bool('1')

**Show me how I search for a ready-made implementation with pytest-nodev.**

First, install the `latest version of pytest-nodev <https://pypi.python.org/pypi/pytest-nodev>`_
from the Python Package Index::

Expand All @@ -58,26 +57,28 @@ decorate it with ``pytest.mark.candidate`` as follows::

Finally, instruct pytest to run your test on all candidate callables in the Python standard library::

$ py.test test_parse_bool.py --candidates-from-stdlib
$ py.test --candidates-from-stdlib test_parse_bool.py
======================= test session starts ==========================
platform darwin -- Python 3.5.0, pytest-2.8.7, py-1.4.31, pluggy-0.3.1
platform darwin -- Python 3.5.1, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /tmp, inifile: setup.cfg
plugins: nodev-1.0.0, timeout-1.0.0
collected 3259 items
collected 4000 items

test_parse_bool.py xxxxxxxxxxxx[...]xxxxxxxxXxxxxxxxx[...]xxxxxxxxxxxx

============================== 1 hit =================================
====================== pytest_nodev: 1 passed ========================

test_parse_bool.py::test_parse_bool[distutils.util:strtobool] PASSED

test_parse_bool.py::test_parse_bool[distutils.util:strtobool] HIT
=== 3999 xfailed, 1 xpassed, 260 pytest-warnings in 75.38 seconds ====

==== 3258 xfailed, 1 xpassed, 27 pytest-warnings in 45.07 seconds ====
In just over a minute pytest-nodev collected 4000 functions from the standard library,
run your specification test on all of them and
reported that the `strtobool`_ function in the distutils.util module
is the only one that passes your test.

In less than a minute pytest-nodev collected more than 3000 functions from the standard library
and run your specification test on all of them and you've got a HIT.
The `strtobool`_ function in the distutils.util module passes the test, so
now you should thoroughly review it and if you like it you may use it in your code,
no need to write your own implementation.
Now you can review it and if you like it you may use it in your code.
No need to write your own implementation!

.. _`strtobool`: https://docs.python.org/3/distutils/apiref.html#distutils.util.strtobool

Expand Down Expand Up @@ -126,7 +127,6 @@ Code quality .. image:: https://api.travis-ci.org/nodev-io/pytest-nodev.svg?bra
.. image:: https://coveralls.io/repos/nodev-io/pytest-nodev/badge.svg?branch=master&service=github
:target: https://coveralls.io/github/nodev-io/pytest-nodev
:alt: Coverage Status on Coveralls
nodev website http://nodev.io
============= ======================


Expand Down
4 changes: 2 additions & 2 deletions docs/concepts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ that is an extension of the well known *test-driven development* or TDD.
The idea is that once the developer has written the tests that define the behaviour of a new
function to a degree sufficient to validate the implementation they are going to write
it is good enough to validate
any implementation. Running the tests on a large set of functions may result in a *hit*, that is
any implementation. Running the tests on a large set of functions may result in a *passed*, that is
a function that already implements their feature.

Due to its nature the approach is better suited for discovering smaller functions
Expand All @@ -73,7 +73,7 @@ Tests validation
Another use for pytest-nodev is, with a bit of additional work, to validate a project test suite.
If a test passes with an unexpected object there are two possibilities,
either the test is not strict enough and allows for false positives and needs to be updated,
or the *hit* is actually a function you could use instead of your implementation.
or the *passed* is actually a function you could use instead of your implementation.


Keywords:
Expand Down
4 changes: 2 additions & 2 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,9 @@
import sphinx.environment
from docutils.utils import get_source_line

def _warn_node(self, msg, node):
def _warn_node(self, msg, node, **kwargs):
if not msg.startswith('nonlocal image URI found:'):
self._warnfunc(msg, '%s:%s' % get_source_line(node))
self._warnfunc(msg, '%s:%s' % get_source_line(node), **kwargs)

sphinx.environment.BuildEnvironment.warn_node = _warn_node

Expand Down
3 changes: 2 additions & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Test-driven source code search for Python.
New to the concept of *test-driven code search*?
Jump to the :doc:`quickstart` for a 2 minutes hands-on overview.
Curious about the technique?
Head over to the :doc:`concepts` section.
Head over to the :doc:`concepts` section
or go through our :doc:`tutorial`.
The :doc:`usersguide` documents pytest-nodev usage in details and
covers a few more examples.

Expand Down
14 changes: 11 additions & 3 deletions docs/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,24 @@ Tutorial

.. warning:: This section is work in progress and there will be areas that are lacking.

Starter kit
-----------
nodev starter kit
-----------------

Use of ``--candidates-from-all`` may be very dangerous
and it is disabled by default.

In order to search safely in all modules we suggest to use docker for OS-level isolation.
To kickstart your advanced usage download the nodev-starter-kit::

To kickstart your advanced usage clone the nodev-starter-kit::

$ git clone https://github.com/nodev-io/nodev-starter-kit.git

or better yet, fork it on GitHub and clone your own fork::

$ git clone https://github.com/YOUR_GITHUB_NAME/nodev-starter-kit.git

Enter the starter kit folder::

$ cd nodev-starter-kit

build the nodev docker image with all module from requirements.txt installed::
Expand Down
21 changes: 21 additions & 0 deletions pytest_nodev/blacklists.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
'pydoc',
'tkinter',
'turtle',
'asyncio',
]

OBJECT_BLACKLIST = [
Expand Down Expand Up @@ -120,4 +121,24 @@
'shutil.rmtree',
'turtle.write_docstringdict',
'multiprocessing.semaphore_tracker:main',

# annoying
'urllib.request:URLopener',
'urllib.request:FancyURLopener',
'urllib.request:urlopen',
'urllib.response:addbase',
'aifc.Error',
'aifc.Aifc_write',
'asyncore:file_dispatcher',
'asyncore:file_wrapper',
'sunau:open',
'sunau:Error',
'sunau:Au_write',
'tempfile:TemporaryFile',
'urllib.robotparser:RobotFileParser',
'wave:Wave_write',
'tempfile:mkdtemp',
'tempfile:mkstemp',
'tempfile:mktemp',
'multiprocessing.util',
]
10 changes: 5 additions & 5 deletions pytest_nodev/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,11 +144,11 @@ def pytest_terminal_summary(terminalreporter):
if not hasattr(terminalreporter.config, '_candidate_index'):
return

hit_state = 'passed' if terminalreporter.config.getoption('candidates_fail') else 'xpassed'
hits = terminalreporter.getreports(hit_state)
terminalreporter.write_sep('=', '%d hit' % len(hits), bold=True)
passed_state = 'passed' if terminalreporter.config.getoption('candidates_fail') else 'xpassed'
passed_reports = terminalreporter.getreports(passed_state)
terminalreporter.write_sep('=', 'pytest_nodev: %d passed' % len(passed_reports), bold=True)
terminalreporter.write_line('')
for report in hits:
for report in passed_reports:
terminalreporter.write(report.nodeid)
terminalreporter.write_line(' HIT', bold=True, green=True)
terminalreporter.write_line(' PASSED', bold=True, green=True)
terminalreporter.write_line('')
10 changes: 5 additions & 5 deletions requirements-docs.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
future==0.15.2
py==1.4.31
pytest==2.9.0
pytest==2.9.2
pytest-timeout==1.0.0
alabaster==0.7.7
Babel==2.2.0
alabaster==0.7.8
Babel==2.3.4
docutils==0.12
Jinja2==2.8
MarkupSafe==0.23
Pygments==2.1.3
pytz==2015.7
pytz==2016.6.1
six==1.10.0
snowballstemmer==1.2.1
Sphinx==1.3.6
Sphinx==1.4.5
sphinx-rtd-theme==0.1.9
10 changes: 5 additions & 5 deletions requirements-tests.txt
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
future==0.15.2
py==1.4.31
pytest==2.9.0
pytest==2.9.2
pytest-timeout==1.0.0
apipkg==1.4
coverage==4.0.3
coverage==4.1
execnet==1.4.1
mccabe==0.4.0
mccabe==0.5.0
pep8==1.7.0
pyflakes==1.1.0
pyflakes==1.2.3
pytest-cache==1.0
pytest-cov==2.2.1
pytest-cov==2.3.0
pytest-flakes==1.0.1
pytest-mccabe==0.1
pytest-pep8==1.0.6
2 changes: 1 addition & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
future==0.15.2
py==1.4.31
pytest==2.9.0
pytest==2.9.2
pytest-timeout==1.0.0
2 changes: 1 addition & 1 deletion tests/test_plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ def test_pytest_terminal_summary(testdir):
'--candidates-from-modules=math',
)
result.stdout.fnmatch_lines([
'*test_factorial*math:factorial*HIT',
'*test_factorial*math:factorial*PASSED',
])
assert result.ret == 0

Expand Down