Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Uncategorized tests should not default to expensive #46

Merged
merged 2 commits into from
Nov 7, 2019

Conversation

jsiirola
Copy link
Contributor

Summary/Motivation:

Currently uncategorized pyutilib.th tests default to "smoke, nightly, expensive". This means that the "expensive" test jobs duplicate all of the tests that the smoke/nightly test jobs have already done, which seems redundant ('all' covers that use case). This PR changes the default so that uncategorized tests are marked as "smoke, nightly," but not expensive. This will reduce the amount of unnecessarily repeated tests in most testing setups.

Changes proposed in this PR:

  • remove "expensive" as a default category

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

  1. I agree my contributions are submitted under the BSD license.
  2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

@jsiirola
Copy link
Contributor Author

This PR is meant to seed a discussion around managing test suites. This is a 1-line PR with a fairly significant impact. As such, I am hoping to hear from all the reviewers before we entertain merging it.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 64.004% when pulling 72ff932 on nightly_not_expensive into 2e25e1d on master.

3 similar comments
@coveralls
Copy link

Coverage Status

Coverage remained the same at 64.004% when pulling 72ff932 on nightly_not_expensive into 2e25e1d on master.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 64.004% when pulling 72ff932 on nightly_not_expensive into 2e25e1d on master.

@coveralls
Copy link

Coverage Status

Coverage remained the same at 64.004% when pulling 72ff932 on nightly_not_expensive into 2e25e1d on master.

@codecov-io
Copy link

codecov-io commented Dec 17, 2018

Codecov Report

Merging #46 into master will not change coverage.
The diff coverage is 0%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master      #46   +/-   ##
=======================================
  Coverage   64.94%   64.94%           
=======================================
  Files          87       87           
  Lines        8726     8726           
=======================================
  Hits         5667     5667           
  Misses       3059     3059
Impacted Files Coverage Δ
pyutilib/th/pyunit.py 47.93% <0%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2e25e1d...72ff932. Read the comment docs.

Copy link
Contributor

@blnicho blnicho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am in favor of this change in default behavior. Reducing some redundancy in our testing infrastructure seems like a win.

@jsiirola jsiirola merged commit 806c795 into master Nov 7, 2019
@jsiirola jsiirola deleted the nightly_not_expensive branch November 7, 2019 16:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants