New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add something like: "--newest-first" #3034

Closed
jedie opened this Issue Dec 14, 2017 · 14 comments

Comments

Projects
None yet
6 participants
@jedie

jedie commented Dec 14, 2017

There is --failed-first ... what's about a --newest-first: Execute the tests from the newest test file first. Sort by file change timestamp.

The idea: If i add a new test, then i would like to see if this new test will be failed first ;)

@pytestbot

This comment has been minimized.

pytestbot commented Dec 14, 2017

GitMate.io thinks the contributors most likely able to help are @nicoddemus, and @RonnyPfannschmidt.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Dec 14, 2017

Thanks @jedie for the suggestion, we appreciate you taking the time to write.

Personally I don't see much value in that option to be honest; when I add a new test file I usually execute it immediately by passing its filename in the command-line, and I never add a bunch of test files and then try to execute the whole suite but would like to see the newly added files first.

This however should be dead-simple to add and is an issue good for a first time contributor, so I don't have anything against adding it to the core. 😉

Perhaps do you have a more specific use case which that option would help that I'm not seeing though?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Dec 14, 2017

@nicoddemus perhaps this is about adding new items in existing files, its not clear how to implement this then, and certainly not easy

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Dec 14, 2017

@nicoddemus perhaps this is about adding new items in existing files, its not clear how to implement this then, and certainly not easy

If that's the case then I agree it seems more useful, although the original text is "Execute the tests from the newest test file first". @jedie can you clarify please?

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Dec 14, 2017

as far as i can tell this one can be done as a plugin - so perhaps thats a good place to experiment with different expressions oft he feature

@jedie

This comment has been minimized.

jedie commented Dec 14, 2017

Sorry for the bad English;)

I mean, the latest added/modified tests should be run first.

We can check the file modify date and maybe we can look for new test names: Each pass saves a list of all tests. When new tests are added, they are sorted first.

Maybe the order can look like this:

  1. "--failed-first" tests
  2. new test names
  3. tests from newest modified files
  4. all other tests
@blueyed

This comment has been minimized.

Contributor

blueyed commented Dec 14, 2017

btw: in this regard it would be useful if there would be a cache for the collected items (test ids with files and their timestamps).
This could speed up the collection in general (only modified and new files would need to be considered), and could be used by a plugin that implements --newest-first etc then.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Dec 14, 2017

Hey @blueyed,

in this regard it would be useful if there would be a cache for the collected items (test ids with files and their timestamps).

You mean add this to the core? I'm not sure, while it would be useful in this special case, I suspect different plugins would have different needs as to what store in the cache (a plugin might be interested to store git hashes instead of timestamps for example). I don't think it is possible to store something that will be useful for plugins in a general manner.

Keep in mind that each plugin can easily save the information on cache themselves, in the exact manner they want.

This could speed up the collection in general

Hmm not sure, why would that speed up the collection? We still need to import the modules and create the items as we can't save those on the cache.

@blueyed

This comment has been minimized.

Contributor

blueyed commented Dec 15, 2017

why would that speed up the collection

Using -k foo might then be able to only import the file defining the matching test items (without collecting everything before) - only checking mtime of all others.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Dec 15, 2017

Oh when -k is used, got it, thanks!

@feuillemorte

This comment has been minimized.

Member

feuillemorte commented Feb 8, 2018

I think we can write another plugin like this:

class LFPlugin(object):
""" Plugin which implements the --lf (run last-failing) option """

for new tests. But I can only understand when file was modified and start to run tests first from this file. But I don't know how to check which exactly test was modified. So, if test file contains 50 tests these tests will collect in default order

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Feb 8, 2018

But I can only understand when file was modified and start to run tests first from this file.

Certainly, but I think that's what the OP has in mind as well.

@feuillemorte

This comment has been minimized.

Member

feuillemorte commented Feb 11, 2018

Ok, then I try to do it :)

feuillemorte added a commit to feuillemorte/pytest that referenced this issue Feb 13, 2018

feuillemorte added a commit to feuillemorte/pytest that referenced this issue Feb 23, 2018

@feuillemorte feuillemorte self-assigned this Feb 27, 2018

nicoddemus added a commit that referenced this issue Mar 2, 2018

@feuillemorte

This comment has been minimized.

Member

feuillemorte commented Apr 8, 2018

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment