Allow index-only test names

[zorchan]

What: I would like to add a feature to ddt so that test names can be generated using just the index only.

How: To allow that to happen, I am making use of kwargs at the class-level decorator @ddt. When decorating a class with @ddt(formatTestName=FormatTestName.INDEX_ONLY), the boolean value will be passed to mk_test_name() and test names will just be index-only.

Why: An easier way to trigger a specific test to run.

Testing: New unit test added. Using tox to run against py27 and py35.

GLOB sdist-make: /path/masked/ddt/setup.py
py27 inst-nodeps: /path/masked/ddt/.tox/.tmp/package/1/ddt-1.3.1.zip
py27 installed: DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support,alabaster==0.7.12,Babel==2.8.0,certifi==2020.4.5.1,chardet==3.0.4,configparser==4.0.2,coverage==5.1,ddt==1.3.1,docutils==0.16,entrypoints==0.3,enum34==1.1.10,flake8==3.7.9,funcsigs==1.0.2,functools32==3.2.3.post2,idna==2.9,imagesize==1.2.0,Jinja2==2.11.2,MarkupSafe==1.1.1,mccabe==0.6.1,mock==3.0.5,nose==1.3.7,packaging==20.3,pycodestyle==2.5.0,pyflakes==2.1.1,Pygments==2.5.2,pyparsing==2.4.7,pytz==2019.3,PyYAML==5.3.1,requests==2.23.0,six==1.14.0,snowballstemmer==2.0.0,Sphinx==1.8.5,sphinxcontrib-programoutput==0.16,sphinxcontrib-websupport==1.1.2,typing==3.7.4.1,urllib3==1.25.8
py27 run-test-pre: PYTHONHASHSEED='2507378467'
py27 run-test: commands[0] | nosetests -s --with-coverage --cover-package=ddt --cover-html
.................................................................................
Name     Stmts   Miss  Cover
----------------------------
ddt.py     131      0   100%
----------------------------------------------------------------------
Ran 81 tests in 0.092s

OK
py27 run-test: commands[1] | flake8 ddt.py test
py27 run-test: commands[2] | sphinx-build -b html docs docs/_build
Running Sphinx v1.8.5
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: [] 0 added, 1 changed, 0 removed
reading sources... [100%] example

/path/masked/ddt/docs/example.rst:29: WARNING: Include file u'/path/masked/ddt/test/test_data_dict_dict.json' not found or reading it failed
/path/masked/ddt/docs/example.rst:34: WARNING: Include file u'/path/masked/ddt/test/test_data_dict_dict.yaml' not found or reading it failed
/path/masked/ddt/docs/example.rst:39: WARNING: Include file u'/path/masked/ddt/test/test_data_dict.json' not found or reading it failed
/path/masked/ddt/docs/example.rst:44: WARNING: Include file u'/path/masked/ddt/test/test_data_dict.yaml' not found or reading it failed
/path/masked/ddt/docs/example.rst:49: WARNING: Include file u'/path/masked/ddt/test/test_data_list.json' not found or reading it failed
/path/masked/ddt/docs/example.rst:54: WARNING: Include file u'/path/masked/ddt/test/test_data_list.yaml' not found or reading it failed
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [ 50%] example
writing output... [100%] index

generating indices... genindex py-modindex
writing additional pages... search
copying static files... WARNING: html_static_path entry u'/path/masked/ddt/docs/_static' does not exist
done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 7 warnings.

The HTML pages are in docs/_build.
py35 inst-nodeps: /path/masked/ddt/.tox/.tmp/package/1/ddt-1.3.1.zip
py35 installed: coverage==5.1,ddt==1.3.1,entrypoints==0.3,flake8==3.7.9,mccabe==0.6.1,nose==1.3.7,pycodestyle==2.5.0,pyflakes==2.1.1,PyYAML==5.3.1,six==1.14.0
py35 run-test-pre: PYTHONHASHSEED='2507378467'
py35 run-test: commands[0] | nosetests -s --with-coverage --cover-package=ddt --cover-html
.................................................................................
Name     Stmts   Miss  Cover
----------------------------
ddt.py     131      0   100%
----------------------------------------------------------------------
Ran 81 tests in 0.099s

OK
py35 run-test: commands[1] | flake8 ddt.py test
___________________________________________________________________________ summary ____________________________________________________________________________
  py27: commands succeeded
  py35: commands succeeded
  congratulations :)

[zorchan]

Hello @wswld , are you the right person to ping? If yes, would you please kindly review this PR? Thank you.

[skeletorXVI]

The opposite would also be great, having the option to only use names and ditch indexes entirely.

[zorchan]

Hi @wswld and @skeletorXVI , thanks for reacting to this pull request. I have a medium priority around this change and would like to receive a green light asap so that the next release will have my changes included. My side wants this change because it will allow easier execution of a specific test.

Given my below suggestion is okay, I can adjust my code here to leave a room for further changes while I will not have the time to implement extra ways to format test names. For example, instead of using a boolean for the kwargs, I can make that to accept an enum:

class FormatTestName(Enum):
    DEFAULT  # e.g. testfunction_1_valueX_valueY
    INDEX_ONLY

@ddt(formatTestName=INDEX_ONLY)
class TestFoo(unittest):
    ...

In such case, adding other ways to format a test name will be driven by the enum. Does that sound okay? In fact, for @skeletorXVI , what use case you are considering so that you don't want the index?

I really appreciate all inputs here and I am looking forward to see further comments from both of you. Thank you.

[skeletorXVI]

I write most of my data-driven unit tests using YAML files. I always name them based on what case they are testing. I am using PyCharm, removing or adding tests the indices change and I have to change all run configurations for the affected tests.

I am considering implementing #66 once I have some time to further improve compatibility with PyCharm.

[wswld]

@zorchan definitely go ahead with adding room for further changes and I will review this ASAP.

[zorchan]

Hi @wswld and @skeletorXVI , a new commit is ready and I also re-ran the tests. Thank you.

[wswld]

Hi, everything looks good aside from a bunch of documentation and naming-related nitpicks.

[zorchan]

A new commit is ready but tox did not work and I just ran the ./build.sh under a virtual env. Thank you.

[wswld]

Merged. Thank you for your contribution.

[zorchan]

Hi @wswld , thanks for accepting the change. After releasing 1.4.0, I saw that there are some problems reported by others.

1. Do you think it is actually better to switch back to the original approach which we should avoid naming a class with prefix test?

2. It seems like enum is also created another dependency problem. Should we be considering using non-enum?

For example:

# These 2 lines completely replaced the enum
# Hence getting rid of `nose` and `enum` dependencies)
DEFAULT_NAME_FORMAT = "default_name_name_format"
INDEX_ONLY_NAME_FORMAT = "index_only_name_format"

To decorate a class:

@ddt(testNameFormat=ddt.INDEX_ONLY_NAME_FORMAT) 

Thank you.

[wswld]

Hi,

No worries, we have a proposed solution for enum problem in #85 and we're going to drop nose dependency for good in #86 in favor of pytest

There is no need for @notest decorator with the latter