Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to sort tests based on information from @small, @medium, and @large #3748

Closed
sebastianbergmann opened this issue Jul 5, 2019 · 5 comments
Assignees

Comments

@sebastianbergmann
Copy link
Owner

@sebastianbergmann sebastianbergmann commented Jul 5, 2019

  • Tests annotated with @small should be run before tests annotated with @medium
  • Tests annotated with @medium should be run before tests annotated with @large
  • Tests annotated with @large should be run before tests not annotated with @small, @medium, or @large
@localheinz
Copy link
Collaborator

@localheinz localheinz commented Jul 5, 2019

Should this be an additional sorting strategy, or the default?

Loading

@sebastianbergmann
Copy link
Owner Author

@sebastianbergmann sebastianbergmann commented Jul 6, 2019

Additional sorting strategy, of course. Sorry for not being clear.

Loading

@sebastianbergmann sebastianbergmann changed the title Sort tests based on information from @small, @medium, and @large Add option to sort tests based on information from @small, @medium, and @large Jul 6, 2019
@epdenouden
Copy link
Collaborator

@epdenouden epdenouden commented Jul 6, 2019

I'll make this tomorrow, if nobody is working on it already. It's a nice little reprieve from the data providers.

Loading

@epdenouden epdenouden self-assigned this Jul 6, 2019
@epdenouden
Copy link
Collaborator

@epdenouden epdenouden commented Jul 6, 2019

Something is wrong with the execution sorting as --order-by=depends,reverse in TestCaseTest. It was an artifact of how TestCaseTest is set up and interaction between some globals-backup behaviour. Has something to do with

final class TestCaseTest extends TestCase
{
    // [...]
    protected $backupGlobalsBlacklist = ['i', 'singleton'];
    // [...]
}

and was fixed by adding a @depends between tests that are testing globals backup in sequence.

Loading

epdenouden added a commit to epdenouden/phpunit that referenced this issue Jul 6, 2019
epdenouden added a commit to epdenouden/phpunit that referenced this issue Jul 6, 2019
@epdenouden
Copy link
Collaborator

@epdenouden epdenouden commented Jul 6, 2019

Let me know if you'd rather have a pull request based on 8.2 :)

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants