New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pitest: create profile for non-checks code #4367

Closed
romani opened this Issue May 22, 2017 · 14 comments

Comments

Projects
3 participants
@romani
Member

romani commented May 22, 2017

right now we do have maven profiles only for Checks (check packages),
we do need create one more(or few) profiles to run tests of non-checks code.

Whole our code should be covered by pitest. Pitest should check all packages of code.
No changes for test are required now, we just need profile(s) and strict mutationThreshold value in profiles to fail build if threshold is lowered.

Execution time of profile should not be more then 15-20 min (limitation of CI for execution time of build).

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach May 22, 2017

Member

@romani How is this different than Issue #3746 ?
It sounds like both are asking for non-check pitests and not 100% enforcement.

Member

rnveach commented May 22, 2017

@romani How is this different than Issue #3746 ?
It sounds like both are asking for non-check pitests and not 100% enforcement.

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani May 22, 2017

Member

This issue does not demand 100% mutation coverage - just create profile and enforce what we have now

Member

romani commented May 22, 2017

This issue does not demand 100% mutation coverage - just create profile and enforce what we have now

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 25, 2017

Contributor

@romani, I found that com.puppycrawl.tools.checkstyle.DefaultContext has no test, should it be included to non-checks profile or ignored?

Contributor

Nimfadora commented May 25, 2017

@romani, I found that com.puppycrawl.tools.checkstyle.DefaultContext has no test, should it be included to non-checks profile or ignored?

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach May 25, 2017

Member

@Nimfadora IMO, it should be included incase there ever comes a time where we expand it's functionality.

Member

rnveach commented May 25, 2017

@Nimfadora IMO, it should be included incase there ever comes a time where we expand it's functionality.

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani May 26, 2017

Member

Ok to add if that is required by smth.

Member

romani commented May 26, 2017

Ok to add if that is required by smth.

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 27, 2017

Contributor

@romani, what should I do with grammar package tests, big part of them use checks, for ex. Java7DiamondTest uses MemberNameCheck. Should I add these tests to profiles which include used Check classes? And second question, there are some classes in test dir, like XdocUtil, CheckUtil, should I include them in targetClasses instead of targetTests?

Contributor

Nimfadora commented May 27, 2017

@romani, what should I do with grammar package tests, big part of them use checks, for ex. Java7DiamondTest uses MemberNameCheck. Should I add these tests to profiles which include used Check classes? And second question, there are some classes in test dir, like XdocUtil, CheckUtil, should I include them in targetClasses instead of targetTests?

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 27, 2017

Contributor

@romani, another issue, JavadocParseTreeTest shows timeout many times and completes in 22 minutes, even if it is alone in profile.

Contributor

Nimfadora commented May 27, 2017

@romani, another issue, JavadocParseTreeTest shows timeout many times and completes in 22 minutes, even if it is alone in profile.

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 27, 2017

Contributor

@romani, and not all checks are covered by mutation tests, annotation package and those in base package: com.puppycrawl.tools.checkstyle.checks.* . What were the reasons to not include them?

Contributor

Nimfadora commented May 27, 2017

@romani, and not all checks are covered by mutation tests, annotation package and those in base package: com.puppycrawl.tools.checkstyle.checks.* . What were the reasons to not include them?

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach May 27, 2017

Member

@Nimfadora It was probably just an oversight they were missed as we wish to enforce pitest over whole project.

Member

rnveach commented May 27, 2017

@Nimfadora It was probably just an oversight they were missed as we wish to enforce pitest over whole project.

Nimfadora pushed a commit to Nimfadora/checkstyle that referenced this issue May 27, 2017

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 27, 2017

Contributor

@romani, add profiles for all files except grammars and internal packages, BaseCheckTestSupport, BaseFileSetCheckTestSupport, BriefUtLogger. Last classes and those in internal package are util classes in test dir mostly. Case with grammar package I discribed in comments before.

Contributor

Nimfadora commented May 27, 2017

@romani, add profiles for all files except grammars and internal packages, BaseCheckTestSupport, BaseFileSetCheckTestSupport, BriefUtLogger. Last classes and those in internal package are util classes in test dir mostly. Case with grammar package I discribed in comments before.

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 28, 2017

Contributor

@rnveach, maybe you could answer my questions about grammar and internal packages, and classes mentioned before?

Contributor

Nimfadora commented May 28, 2017

@rnveach, maybe you could answer my questions about grammar and internal packages, and classes mentioned before?

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach May 28, 2017

Member

@Nimfadora You didnt ask any questions as there is no ? in your post so it is unclear what is not a statement.
We should be creating no pitest for the test tier and the grammar package. Everything else should be connected to pitest. Pitest is only for production code in main folder.

Member

rnveach commented May 28, 2017

@Nimfadora You didnt ask any questions as there is no ? in your post so it is unclear what is not a statement.
We should be creating no pitest for the test tier and the grammar package. Everything else should be connected to pitest. Pitest is only for production code in main folder.

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora May 28, 2017

Contributor

@rnveach, #4367 (comment) , here I asked some questions, but you answered them in this comment, thank you.

Contributor

Nimfadora commented May 28, 2017

@rnveach, #4367 (comment) , here I asked some questions, but you answered them in this comment, thank you.

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora pushed a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora pushed a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora pushed a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora pushed a commit to Nimfadora/checkstyle that referenced this issue May 29, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue May 31, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue May 31, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Jul 4, 2017

Member

all profiles are created

Member

romani commented Jul 4, 2017

all profiles are created

@romani romani closed this Jul 4, 2017

@romani romani added this to the 8.1 milestone Jul 4, 2017

@romani romani moved this from In Progress to Done in Practice What You Preach Jul 4, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment