New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pitest: increase mutation coverage for pitest-check-annotation profile to 100% #4402

Closed
romani opened this Issue May 31, 2017 · 7 comments

Comments

Projects
2 participants
@romani
Member

romani commented May 31, 2017

We should increase coverage for pitest-check-anotation profile up to 100%.
This issue is a subtask of #3708

Current threshold of pitest-checkstyle-annotation profile: XX

ATTENTION:
When I simply add com.puppycrawl.tools.checkstyle.checks.annotation.* build fails:
Failed to execute goal org.pitest:pitest-maven:1.2.0:mutationCoverage (default-cli) on project checkstyle: Execution default-cli of goal org.pitest:pitest-maven:1.2.0:mutationCoverage failed: 65535

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora Jun 1, 2017

Contributor

@romani it seems that I've catched this bug #297 while adding annotation profile. When I simply add com.puppycrawl.tools.checkstyle.checks.annotation.* build fails with the common error, but when I add all of the files in package separately, build passes. Magic. Should I submit first variant with the package as it may pass on CI normally or add all classes separately to be confident that profile will pass?

Contributor

Nimfadora commented Jun 1, 2017

@romani it seems that I've catched this bug #297 while adding annotation profile. When I simply add com.puppycrawl.tools.checkstyle.checks.annotation.* build fails with the common error, but when I add all of the files in package separately, build passes. Magic. Should I submit first variant with the package as it may pass on CI normally or add all classes separately to be confident that profile will pass?

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Jun 1, 2017

Member

add all classes separately to be confident that profile will pass?

it is better to have profile that pass on CI.

build fails with the common error

please share full stacktrace, I do not understand what is a problem.
You referencing an old issue .... I do not understand why.

Member

romani commented Jun 1, 2017

add all classes separately to be confident that profile will pass?

it is better to have profile that pass on CI.

build fails with the common error

please share full stacktrace, I do not understand what is a problem.
You referencing an old issue .... I do not understand why.

@Nimfadora

This comment has been minimized.

Show comment
Hide comment
@Nimfadora

Nimfadora Jun 2, 2017

Contributor
VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.121-b13
Uptime : 41656
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.1/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.1
BootClassPathSupported : true

        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:103)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:49)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:112)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:103)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:45)
        at org.pitest.maven.RunPitStrategy.execute(RunPitStrategy.java:35)
        at org.pitest.maven.AbstractPitMojo.analyse(AbstractPitMojo.java:407)
        at org.pitest.maven.AbstractPitMojo.execute(AbstractPitMojo.java:351)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
        ... 20 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 65535
        at org.pitest.reloc.asm.ClassReader.readLabel(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.a(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.a(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.b(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.accept(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.accept(Unknown Source)
        at org.pitest.classinfo.ClassInfoVisitor.getClassInfo(ClassInfoVisitor.java:46)
        at org.pitest.classinfo.Repository.nameToClassInfo(Repository.java:70)
        at org.pitest.classinfo.Repository.fetchClass(Repository.java:60)
        at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:16)
        at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:6)
        at org.pitest.functional.FCollection.flatMapTo(FCollection.java:56)
        at org.pitest.functional.FCollection.flatMap(FCollection.java:66)
        at org.pitest.classpath.CodeSource.getTests(CodeSource.java:58)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:80)
        ... 28 more

@romani , This exception occurs when I try to execute annotation profile of such configuration:

<profile>
      <id>pitest-checks-annotation</id>
      <properties>
        <skipTests>true</skipTests>
      </properties>
      <build>
        <plugins>
          <plugin>
            <groupId>org.pitest</groupId>
            <artifactId>pitest-maven</artifactId>
            <version>${pitest.plugin.version}</version>
            <configuration>
              <targetClasses>
                <param>com.puppycrawl.tools.checkstyle.checks.annotation.*</param>
              </targetClasses>
              <targetTests>
                <param>com.puppycrawl.tools.checkstyle.checks.annotation.*</param>
              </targetTests>
              <mutationThreshold>99</mutationThreshold>
              <timeoutFactor>${pitest.plugin.timeout.factor}</timeoutFactor>
              <timeoutConstant>${pitest.plugin.timeout.constant}</timeoutConstant>
              <threads>${pitest.plugin.threads}</threads>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

Sorry, linked issue is really invalid, I haven't noticed it's not a checkstyle repo as saw familiar name:
hcoles/pitest#302 here it is

Contributor

Nimfadora commented Jun 2, 2017

VM : Java HotSpot(TM) 64-Bit Server VM
Vendor : Oracle Corporation
Version : 25.121-b13
Uptime : 41656
Input -> 
 1 : -Dclassworlds.conf=/usr/local/apache-maven-3.2.1/bin/m2.conf
 2 : -Dmaven.home=/usr/local/apache-maven-3.2.1
BootClassPathSupported : true

        at org.pitest.util.Unchecked.translateCheckedException(Unchecked.java:20)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:103)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:49)
        at org.pitest.mutationtest.tooling.MutationCoverage.runReport(MutationCoverage.java:112)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:103)
        at org.pitest.mutationtest.tooling.EntryPoint.execute(EntryPoint.java:45)
        at org.pitest.maven.RunPitStrategy.execute(RunPitStrategy.java:35)
        at org.pitest.maven.AbstractPitMojo.analyse(AbstractPitMojo.java:407)
        at org.pitest.maven.AbstractPitMojo.execute(AbstractPitMojo.java:351)
        at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:133)
        ... 20 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 65535
        at org.pitest.reloc.asm.ClassReader.readLabel(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.a(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.a(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.b(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.accept(Unknown Source)
        at org.pitest.reloc.asm.ClassReader.accept(Unknown Source)
        at org.pitest.classinfo.ClassInfoVisitor.getClassInfo(ClassInfoVisitor.java:46)
        at org.pitest.classinfo.Repository.nameToClassInfo(Repository.java:70)
        at org.pitest.classinfo.Repository.fetchClass(Repository.java:60)
        at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:16)
        at org.pitest.classinfo.NameToClassInfo.apply(NameToClassInfo.java:6)
        at org.pitest.functional.FCollection.flatMapTo(FCollection.java:56)
        at org.pitest.functional.FCollection.flatMap(FCollection.java:66)
        at org.pitest.classpath.CodeSource.getTests(CodeSource.java:58)
        at org.pitest.coverage.execute.DefaultCoverageGenerator.calculateCoverage(DefaultCoverageGenerator.java:80)
        ... 28 more

@romani , This exception occurs when I try to execute annotation profile of such configuration:

<profile>
      <id>pitest-checks-annotation</id>
      <properties>
        <skipTests>true</skipTests>
      </properties>
      <build>
        <plugins>
          <plugin>
            <groupId>org.pitest</groupId>
            <artifactId>pitest-maven</artifactId>
            <version>${pitest.plugin.version}</version>
            <configuration>
              <targetClasses>
                <param>com.puppycrawl.tools.checkstyle.checks.annotation.*</param>
              </targetClasses>
              <targetTests>
                <param>com.puppycrawl.tools.checkstyle.checks.annotation.*</param>
              </targetTests>
              <mutationThreshold>99</mutationThreshold>
              <timeoutFactor>${pitest.plugin.timeout.factor}</timeoutFactor>
              <timeoutConstant>${pitest.plugin.timeout.constant}</timeoutConstant>
              <threads>${pitest.plugin.threads}</threads>
            </configuration>
          </plugin>
        </plugins>
      </build>
    </profile>

Sorry, linked issue is really invalid, I haven't noticed it's not a checkstyle repo as saw familiar name:
hcoles/pitest#302 here it is

romani added a commit that referenced this issue Jun 3, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Jul 25, 2017

Nimfadora added a commit to Nimfadora/checkstyle that referenced this issue Jul 26, 2017

romani added a commit that referenced this issue Jul 26, 2017

@romani romani added this to the 8.2 milestone Jul 26, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Jul 26, 2017

Member

fix is merged

Member

romani commented Jul 26, 2017

fix is merged

@romani romani closed this Jul 26, 2017

@romani romani moved this from To Do to Done in Practice What You Preach Jul 27, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Aug 28, 2017

Member

@Nimfadora ,

from #5015 .

there is regression:
Survived items:target/pit-reports/201708281645/com.puppycrawl.tools.checkstyle.utils/JavadocUtils.java.html:<td class='survived'> target/pit-reports/201708281645/com.puppycrawl.tools.checkstyle.utils/JavadocUtils.java.html:<td class='covered'><pre><span class='survived'> else if (id &#62; TOKEN_VALUE_TO_NAME.length - 1) {</span></pre></td></tr>

please help us to fix.

Member

romani commented Aug 28, 2017

@Nimfadora ,

from #5015 .

there is regression:
Survived items:target/pit-reports/201708281645/com.puppycrawl.tools.checkstyle.utils/JavadocUtils.java.html:<td class='survived'> target/pit-reports/201708281645/com.puppycrawl.tools.checkstyle.utils/JavadocUtils.java.html:<td class='covered'><pre><span class='survived'> else if (id &#62; TOKEN_VALUE_TO_NAME.length - 1) {</span></pre></td></tr>

please help us to fix.

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Sep 1, 2017

Member

looks like we do not have survived any more.
should be fixed after #5045 is merged.

Member

romani commented Sep 1, 2017

looks like we do not have survived any more.
should be fixed after #5045 is merged.

romani added a commit that referenced this issue Sep 1, 2017

@romani romani modified the milestones: 8.3, 8.2 Sep 1, 2017

@romani

This comment has been minimized.

Show comment
Hide comment
@romani

romani Sep 1, 2017

Member

Final fix is merged

Member

romani commented Sep 1, 2017

Final fix is merged

@romani romani closed this Sep 1, 2017

@rnveach rnveach moved this from To Do to Done in Practice What You Preach Oct 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment