Warn if py.test -k foo finds no tests.

[pytestbot]

Originally reported by: Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler)

---

I think it would be a good useability improvement, if py.test would
warn me, if py.test -k foo finds no test at all.

I use version 2.5.2 and I see a green line, which indicates that all is good.

But if no tests was found, at least for me (interactive usage), it is a fault.

It would be nice to see a different color (not green) if no test was selected.

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/500

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

And would you prefer red or something else?

[pytestbot]

Original comment by Jurko Gospodnetić (BitBucket: jurko, GitHub: jurko):

---

+1

That could be flagged as an error (red) or a warning (yellow?). I guess asking for no tests to be run is not something you do on purpose. 😄

Note though that this is different from finding some tests but deciding to skip them, which should be definitely be considered a success.

[pytestbot]

Original comment by Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler):

---

I would prefer a warning color. AFAIK you use yellow.

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

So surely then having 0 collected tests should be yellow too? Currently one gets green for that too.

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

See https://bitbucket.org/hpk42/pytest/pull-request/156/

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

So I've rejected my own pull request. Given the other issue where changing the colour of the status bar was considered which was rejected as well. After some discussion the conclusion there was that the summary line should only ever be green and red, and red should only occur if there's an actual failure. Introducing more colours in general is something we don't really want to do either. In fact there was some discussion of changing colours to abstract things like "success", "error", "warning" but that would be an other issue.

[pytestbot]

Original comment by Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler):

---

Dear Floris, can you give me a hint how to implement this feature in a plugin?

With feature I mean: Display a warning if "-k" selects no test.

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

Hi Floris, maybe we had a misunderstanding. The red/green distinction on the summary line referred to the issue of xfail/xpass which i think should not include the color. If no tests were run at all, i think we can consider showing an uncolored line because nothing either failed (red) or passed (green).

Thomas, i guess you could implement something with the pytest_terminal_summary hook but need to run now so can't give example.

[pytestbot]

Original comment by Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

If we want to do it in the core we can as well adapt PR 156

[pytestbot]

Original comment by Anatoly Bubenkov (BitBucket: bubenkoff, GitHub: bubenkoff):

---

Recently we've got situation like:

#!sh

[10.0.31.34] linux2 Python 2.6.9 cwd: /home/unstable/pytest_jenkins_Paylogic_Mergekeepers
00:05:10 
[10.0.31.28] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.28] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.29] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.29] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.30] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.31] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.31] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.32] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.33] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.33] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:10 
[10.0.31.33] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:14 
[10.0.31.34] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:14 
[10.0.31.34] Python 2.6.9 (default, Oct 30 2013, 17:13:59)  -- [GCC 4.6.3]
00:05:55 10.0.31.28 [3640] / 10.0.31.28 [3640] / 10.0.31.29 [3640] / 10.0.31.29 [3640] / 10.0.31.30 [3640] / 10.0.31.31 [3640] / 10.0.31.31 [3640] / 10.0.31.32 [3640] / 10.0.31.33 [3640] / 10.0.31.33 [3640] / 10.0.31.33 [3640] / 10.0.31.34 [3640] / 10.0.31.34 [3640]
00:05:55 
00:05:55 scheduling tests via LoadScheduling
00:05:56 
00:05:56  generated xml file: /var/lib/jenkins/workspace/Paylogic_Mergekeepers/TEST-results.xml 
00:05:56  generated json file: /var/lib/jenkins/workspace/Paylogic_Mergekeepers/cucumber/cucumber.json 
00:05:56 ==============================  in 590.47 seconds ==============================
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:
00:05:56 Unhandled exception in thread started by 
00:05:56 Error in sys.excepthook:
00:05:56 
00:05:56 Original exception was:

so no tests were run at all (because of the exception)
but py.test exited with 0!
i think it should be considered as a failure and exit code should be non-zero if no tests were ran at all, doesn't matter if it's result of the -k or the collection without the filter
In our case, exiting with 0 when no tests were executed can cause serious issue as we use continuous integration and the breaking feature branch can be automatically merged because of this, i would say, bug
What others think?

[pytestbot]

Original comment by holger krekel (BitBucket: hpk42, GitHub: hpk42):

---

OK, on twitter people also suggest to turn "no tests collected/ran" into an error. https://twitter.com/hpk42/status/519402228118216704
So i am fine with that. @flub could you do a fresh PR? Can't find your old PR156.

[pytestbot]

Original comment by Eric Siegerman (BitBucket: eks, GitHub: eks):

---

See pull request #304. Comments there explain my choice of colours; not sure whether that's better discussed there or here...

[pytestbot]

Original comment by Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler):

---

Would be very nice if "py.test -k foo" finds no test you get some visual feedback. I want some visual feedback telling me: ** no tests found **

[pytestbot]

Original comment by Eric Siegerman (BitBucket: eks, GitHub: eks):

---

Could do; wouldn't be hard at all technically.

I've been using (a more primitive version of) pull request #304 for quite a while, and I like it a lot. How would people feel about that and an explicit message?

[pytestbot]

Original comment by Bruno Oliveira (BitBucket: nicoddemus, GitHub: nicoddemus):

---

I haven't tried it, but I like the solution. :)