Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add something like: "--newest-first" #3034

Closed
jedie opened this issue Dec 14, 2017 · 14 comments
Closed

Add something like: "--newest-first" #3034

jedie opened this issue Dec 14, 2017 · 14 comments

Comments

@jedie
Copy link

@jedie jedie commented Dec 14, 2017

There is --failed-first ... what's about a --newest-first: Execute the tests from the newest test file first. Sort by file change timestamp.

The idea: If i add a new test, then i would like to see if this new test will be failed first ;)

@pytestbot
Copy link
Contributor

@pytestbot pytestbot commented Dec 14, 2017

GitMate.io thinks the contributors most likely able to help are @nicoddemus, and @RonnyPfannschmidt.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Dec 14, 2017

Thanks @jedie for the suggestion, we appreciate you taking the time to write.

Personally I don't see much value in that option to be honest; when I add a new test file I usually execute it immediately by passing its filename in the command-line, and I never add a bunch of test files and then try to execute the whole suite but would like to see the newly added files first.

This however should be dead-simple to add and is an issue good for a first time contributor, so I don't have anything against adding it to the core. 😉

Perhaps do you have a more specific use case which that option would help that I'm not seeing though?

@RonnyPfannschmidt
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt commented Dec 14, 2017

@nicoddemus perhaps this is about adding new items in existing files, its not clear how to implement this then, and certainly not easy

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Dec 14, 2017

@nicoddemus perhaps this is about adding new items in existing files, its not clear how to implement this then, and certainly not easy

If that's the case then I agree it seems more useful, although the original text is "Execute the tests from the newest test file first". @jedie can you clarify please?

@RonnyPfannschmidt
Copy link
Member

@RonnyPfannschmidt RonnyPfannschmidt commented Dec 14, 2017

as far as i can tell this one can be done as a plugin - so perhaps thats a good place to experiment with different expressions oft he feature

@jedie
Copy link
Author

@jedie jedie commented Dec 14, 2017

Sorry for the bad English;)

I mean, the latest added/modified tests should be run first.

We can check the file modify date and maybe we can look for new test names: Each pass saves a list of all tests. When new tests are added, they are sorted first.

Maybe the order can look like this:

  1. "--failed-first" tests
  2. new test names
  3. tests from newest modified files
  4. all other tests
@blueyed
Copy link
Contributor

@blueyed blueyed commented Dec 14, 2017

btw: in this regard it would be useful if there would be a cache for the collected items (test ids with files and their timestamps).
This could speed up the collection in general (only modified and new files would need to be considered), and could be used by a plugin that implements --newest-first etc then.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Dec 14, 2017

Hey @blueyed,

in this regard it would be useful if there would be a cache for the collected items (test ids with files and their timestamps).

You mean add this to the core? I'm not sure, while it would be useful in this special case, I suspect different plugins would have different needs as to what store in the cache (a plugin might be interested to store git hashes instead of timestamps for example). I don't think it is possible to store something that will be useful for plugins in a general manner.

Keep in mind that each plugin can easily save the information on cache themselves, in the exact manner they want.

This could speed up the collection in general

Hmm not sure, why would that speed up the collection? We still need to import the modules and create the items as we can't save those on the cache.

@blueyed
Copy link
Contributor

@blueyed blueyed commented Dec 15, 2017

why would that speed up the collection

Using -k foo might then be able to only import the file defining the matching test items (without collecting everything before) - only checking mtime of all others.

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Dec 15, 2017

Oh when -k is used, got it, thanks!

@feuillemorte
Copy link
Member

@feuillemorte feuillemorte commented Feb 8, 2018

I think we can write another plugin like this:

class LFPlugin(object):
""" Plugin which implements the --lf (run last-failing) option """

for new tests. But I can only understand when file was modified and start to run tests first from this file. But I don't know how to check which exactly test was modified. So, if test file contains 50 tests these tests will collect in default order

@nicoddemus
Copy link
Member

@nicoddemus nicoddemus commented Feb 8, 2018

But I can only understand when file was modified and start to run tests first from this file.

Certainly, but I think that's what the OP has in mind as well.

@feuillemorte
Copy link
Member

@feuillemorte feuillemorte commented Feb 11, 2018

Ok, then I try to do it :)

feuillemorte added a commit to feuillemorte/pytest that referenced this issue Feb 13, 2018
feuillemorte added a commit to feuillemorte/pytest that referenced this issue Feb 23, 2018
@feuillemorte feuillemorte self-assigned this Feb 27, 2018
nicoddemus added a commit that referenced this issue Mar 2, 2018
#3034 Added new option "--new-first"
@feuillemorte
Copy link
Member

@feuillemorte feuillemorte commented Apr 8, 2018

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants