regrtest: add switch -c to run only modified tests

[birkenfeld]

BPO	22806
Nosy	@birkenfeld, @pitrou, @vstinner, @ezio-melotti, @serhiy-storchaka
Files	regrtest_changed.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-10-02.21:15:51.100>
created_at = <Date 2014-11-06.14:59:16.738>
labels = ['type-feature', 'tests']
title = 'regrtest: add switch -c to run only modified tests'
updated_at = <Date 2015-10-02.22:48:59.180>
user = 'https://github.com/birkenfeld'

bugs.python.org fields:

activity = <Date 2015-10-02.22:48:59.180>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2015-10-02.21:15:51.100>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2014-11-06.14:59:16.738>
creator = 'georg.brandl'
dependencies = []
files = ['37138']
hgrepos = []
issue_num = 22806
keywords = ['patch']
message_count = 7.0
messages = ['230748', '230750', '230751', '230752', '252161', '252170', '252171']
nosy_count = 6.0
nosy_names = ['georg.brandl', 'pitrou', 'vstinner', 'ezio.melotti', 'python-dev', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue22806'
versions = ['Python 3.4', 'Python 3.5']

[birkenfeld]

A quick way to select only tests that are modified in the checkout.

[serhiy-storchaka]

Unfortunately this doesn't work with tests for pickle, json or tkinter.

I'm afraid this feature would make false promise. Without careful check you can not be confident that all modified tests are selected. If you going to add this switch, it should emit a

---

•              *
  

• LOUD WARNING *

•              *
  

---

and enumerate all modified files containing 'test' in its path which are not recognized as valid test name.

[pitrou]

I agree this is more dangerous than useful.
(even if you haven't modified a test it may still be impacted by some other change)

[birkenfeld]

Well, this is not meant as a comprehensive "run ALL impacted tests" because that is impossible in general :)

An alternate suggestion would be to allow filenames like "Lib/test/test_foo.py" as arguments to regrtest. Then I could run without the switch using

.../regrtest.py hg status -amn

[vstinner]

I'm now closing this issue because Antoine, Serhiy and me dislike the idea.

An alternate suggestion would be to allow filenames like "Lib/test/test_foo.py" as arguments to regrtest.

This sounds like a better plan. Please open a new issue if you still need that.

FYI I refactoring regrtest.py into libregrtest, it might be easier to enhance it now.

[python-dev]

New changeset 1005573e6a74 by Victor Stinner in branch 'default':
Issue bpo-22806: Add python -m test --list-tests command to list tests.
https://hg.python.org/cpython/rev/1005573e6a74

[vstinner]

I added "python -m test --list-tests". It's not perfect but it's better than nothing :-p