New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pitest: increase mutation coverage for pitest-checks-coding profile to 100% #5008

Open
Nimfadora opened this Issue Aug 25, 2017 · 9 comments

Comments

Projects
None yet
3 participants
@Nimfadora
Contributor

Nimfadora commented Aug 25, 2017

We should increase coverage for pitest-checks-coding profile up to 100%.
This issue is a subtask of #3708

Current threshold of pitest-checks-coding profile: 95

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 11, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 23, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 23, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 24, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Sep 24, 2017

romani added a commit that referenced this issue Sep 24, 2017

romani added a commit that referenced this issue Sep 24, 2017

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Dec 6, 2017

Member

I am working on this.

Member

rnveach commented Dec 6, 2017

I am working on this.

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Dec 6, 2017

Member

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/FallThroughCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@2ae499cb_400
This item isn't being killed because Comment.intersects is too lenient on checking absolute positions. We can give a longer range then the comment is and still get the comment back. We can't add a secondary comment to confuse it because once we find one that hits, we use that and exit the check.

Member

rnveach commented Dec 6, 2017

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/FallThroughCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@2ae499cb_400
This item isn't being killed because Comment.intersects is too lenient on checking absolute positions. We can give a longer range then the comment is and still get the comment back. We can't add a secondary comment to confuse it because once we find one that hits, we use that and exit the check.

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 6, 2017

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 6, 2017

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Dec 6, 2017

Member

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/HiddenFieldCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@6a13d885_231
This item isn't killed because we are always sending processVariable a lambda token when the method only deals with local variables or parameter definitions. Even changing it to give the method the parameter definition of the lambdas didn't produce any differences in UTs.
We will need to identify what the correct coding is here and find a way to create a violation for it.

Member

rnveach commented Dec 6, 2017

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/HiddenFieldCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@6a13d885_231
This item isn't killed because we are always sending processVariable a lambda token when the method only deals with local variables or parameter definitions. Even changing it to give the method the parameter definition of the lambdas didn't produce any differences in UTs.
We will need to identify what the correct coding is here and find a way to create a violation for it.

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 6, 2017

romani added a commit that referenced this issue Dec 7, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Dec 7, 2017

Member

Remember that code in Check is likely to be changed to be more exact and strict.

Member

romani commented Dec 7, 2017

Remember that code in Check is likely to be changed to be more exact and strict.

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Dec 7, 2017

Member

@romani Those changes require more than a simple change. They are posted to you to confirm how you want them changed.
HiddenCheck could be changed to send processVariable the right token, but I'm not sure if I could make that pass pitest unless your saying the logic of the check doesn't need it and I can remove it all together.
For FallthroughCheck, are we changing Comment.intersects which is API, or are we changing the check to not give the extra addition/subtraction?

Member

rnveach commented Dec 7, 2017

@romani Those changes require more than a simple change. They are posted to you to confirm how you want them changed.
HiddenCheck could be changed to send processVariable the right token, but I'm not sure if I could make that pass pitest unless your saying the logic of the check doesn't need it and I can remove it all together.
For FallthroughCheck, are we changing Comment.intersects which is API, or are we changing the check to not give the extra addition/subtraction?

@rnveach

This comment has been minimized.

Show comment
Hide comment
@rnveach

rnveach Dec 7, 2017

Member

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/IllegalTypeCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@13c96569_316
This item isn't killed because nothing bad happens from allowing star imports. Class name is erronously made java.util. for java.util.*. It never gets added to the search queue so it is never found.
I don't know how to fix this other than the fact we remove the check completely or use alot of reflection.

Member

rnveach commented Dec 7, 2017

http://rveach.no-ip.org/checkstyle/regression/pitest-reports/5/com.puppycrawl.tools.checkstyle.checks.coding/IllegalTypeCheck.java.html#org.pitest.mutationtest.report.html.SourceFile@13c96569_316
This item isn't killed because nothing bad happens from allowing star imports. Class name is erronously made java.util. for java.util.*. It never gets added to the search queue so it is never found.
I don't know how to fix this other than the fact we remove the check completely or use alot of reflection.

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 7, 2017

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 8, 2017

romani added a commit that referenced this issue Dec 8, 2017

rnveach added a commit to rnveach/checkstyle that referenced this issue Dec 8, 2017

romani added a commit that referenced this issue Dec 8, 2017

timurt added a commit to timurt/checkstyle that referenced this issue Dec 19, 2017

timurt added a commit to timurt/checkstyle that referenced this issue Dec 19, 2017

timurt added a commit to timurt/checkstyle that referenced this issue Dec 19, 2017

romani added a commit that referenced this issue Feb 25, 2018

romani added a commit that referenced this issue Feb 25, 2018

@romani romani removed the GSoC2017 label Mar 11, 2018

@romani romani removed this from To Do in Practice What You Preach Mar 11, 2018

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Mar 31, 2018

Member

current level is https://github.com/checkstyle/checkstyle/blob/master/pom.xml#L1841 :
<mutationThreshold>98</mutationThreshold>

current uncovered mutations are at: https://github.com/checkstyle/checkstyle/blob/master/.ci/shippable.sh#L54

Member

romani commented Mar 31, 2018

current level is https://github.com/checkstyle/checkstyle/blob/master/pom.xml#L1841 :
<mutationThreshold>98</mutationThreshold>

current uncovered mutations are at: https://github.com/checkstyle/checkstyle/blob/master/.ci/shippable.sh#L54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment