Skip to content

CommandAPI code coverage reports

Jump to bottom
Jorel Ali edited this page May 28, 2023 · 2 revisions

The CommandAPI project has been structured to allow you to run code coverage reports against the main CommandAPI code. A code coverage report shows what lines of code are run during program execution and helps identify portions of code which haven't been run or tested, in other words, it shows what code has been "covered" by tests. Ideally, all paths that should be runnable should be covered, but we're not enforcing any strict rules on the required code coverage for the CommandAPI.

How to run code coverage tests (Bukkit)

For Bukkit, the CommandAPI uses JaCoCo for code coverage, alongside the Bukkit testing matrix. Known code coverage is as listed:

  • commandapi-core
  • commandapi-bukkit-core
  • Your chosen NMS instance and its parents. For example, if you chose to test against Minecraft 1.19.4, the following modules will appear under code coverage:
    • commandapi-bukkit-1.19.4
    • commandapi-bukkit-nms-common

To run code coverage, run the following command, using mojang-mappings for 1.19.4:

mvn clean verify -Dmaven.javadoc.skip=true -T 6 -P Platform.Bukkit,Minecraft_1_19_4_Mojang -am

If your computer is struggling to run the above command, remove -T 6. This flag tells Maven to run across 6 threads for a parallel build which should make building faster, but can sometimes be a bit unreliable (causing builds to fail for absolutely no reason).

The -am flag shouldn't be necessary (all it does is tell Maven to build all dependencies of every module, which it should be doing by default anyway)

Mojang-mapped tests for code coverage

Code coverage tests must be run against Mojang-mapped code, so the Mojang-mapped code profile is required, e.g. Minecraft_1_19_4_Mojang. Profiles will always be of the form:

Minecraft_X_Y_Z_Mojang

or

Minecraft_X_Y_Mojang

Where:

  • X is the major version
  • Y is the minor version
  • Z is the patch version

For example:

  • 1.17 -> Minecraft_1_17_Mojang
  • 1.19.4 -> Minecraft_1_19_4_Mojang

At the time of writing, we only have code coverage reports for 1.19.4 NMS.

How to add code coverage to a new NMS module

Add the following to your <build><plugins> section of your pom.xml file in your commandapi-bukkit-X.Y.Z module:

<!-- Allow code coverage -->
<plugin>
	<groupId>org.jacoco</groupId>
	<artifactId>jacoco-maven-plugin</artifactId>
	<executions>
		<execution>
			<id>default-prepare-agent</id>
			<goals>
				<goal>prepare-agent</goal>
			</goals>
		</execution>
		<execution>
			<id>report</id>
			<phase>verify</phase>
			<goals>
				<goal>report</goal>
			</goals>
		</execution>
	</executions>
</plugin>

Add a new profile for it in the pom.xml for commandapi-bukkit-test-tests to allow mojang-mapped switching. You may also want to add a basic profile for non-mojang-mapped code to be able to run tests against non-mojang-mapped code:

<profile>
	<!-- Non-mojang-mapped profile -->
	<id>Minecraft_1_19_4</id>
	<dependencies>
		<dependency>
			<!-- Needed for 1.19.4 Brigadier logging. This is also present in
			 commandapi-core, but isn't inherited here automatically, so we add
			 it here -->
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-api</artifactId>
			<version>2.19.0</version>
			<scope>provided</scope>
		</dependency>
		<dependency>
			<groupId>dev.jorel</groupId>
			<artifactId>commandapi-bukkit-shade</artifactId>
			<version>${project.version}</version>
		</dependency>
		<dependency>
			<groupId>dev.jorel</groupId>
			<artifactId>commandapi-bukkit-test-impl-1.19.4</artifactId>
			<version>${project.version}</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.spigotmc</groupId>
			<artifactId>spigot</artifactId>
			<version>1.19.4-R0.1-SNAPSHOT</version>
		</dependency>
		<dependency>
			<groupId>io.papermc.paper</groupId>
			<artifactId>paper-api</artifactId>
			<version>1.19.4-R0.1-SNAPSHOT</version>
		</dependency>
	</dependencies>
</profile>
<profile>
	<!-- Mojang-mapped profile -->
	<id>Minecraft_1_19_4_Mojang</id>
	<dependencies>
		<dependency>
			<!-- Needed for 1.19.4 Brigadier logging. This is also present in
			 commandapi-core, but isn't inherited here automatically, so we add
			 it here -->
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-api</artifactId>
			<version>2.19.0</version>
			<scope>provided</scope>
		</dependency>
		<!-- Mojang-mapped profile needs to be linked against mojang-mapped NMS -->
		<dependency>
			<groupId>dev.jorel</groupId>
			<artifactId>commandapi-bukkit-nms-dependency-mojang-mapped</artifactId>
			<version>${project.version}</version>
			<type>pom</type>
		</dependency>
		<dependency>
			<groupId>dev.jorel</groupId>
			<artifactId>commandapi-bukkit-shade</artifactId>
			<version>${project.version}</version>
		</dependency>
		<dependency>
			<groupId>dev.jorel</groupId>
			<artifactId>commandapi-bukkit-test-impl-1.19.4</artifactId>
			<version>${project.version}</version>
			<!-- Mojang-mapped code needs to be linked against mojang-mapped classifier for test implementation -->
			<classifier>mojang-mapped</classifier>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.spigotmc</groupId>
			<artifactId>spigot</artifactId>
			<version>1.19.4-R0.1-SNAPSHOT</version>
			<classifier>remapped-mojang</classifier>
			<scope>provided</scope>
		</dependency>
		<dependency>
			<groupId>io.papermc.paper</groupId>
			<artifactId>paper-api</artifactId>
			<version>1.19.4-R0.1-20230320.085443-20</version>
		</dependency>
	</dependencies>
</profile>

Ensure your mojang-mapped version is present under commandapi-bukkit-nms-dependency-mojang-mapped's pom.xml file.

How to view/interpret code coverage reports (in your browser)

Code coverage reports can be viewed in your browser by opening commandapi-codecov/target/site/jacoco-aggregate/index.html. A basic code coverage report will look like this:

image

This shows that commandapi-bukkit-core, commandapi-core, commandapi-bukkit-1.19.4 and commandapi-bukkit-nms-common were all "picked up" in the code coverage run and have meaningful reports. It shows that about 55% of commandapi-bukkit-core was run during the running of commandapi-bukkit-test-tests. By clicking on the element you want to inspect, you can find out more information about it.

For example, clicking on commandapi-bukkit-core will show the various Java packages and their breakdown of code coverage:

image

If we go to dev.jorel.commandapi.arguments, we can inspect classes:

image

If you click on a class, you can see a breakdown of methods that were called. For example, if we look at LocationArgument, we get this:

image

From this, we can see that the getPrimitiveType(), getArgumentType() and getLocationType methods were not run. We can also see that various lambdas are never run, and the LocationArgument(String) constructor is never called. getLocationType() states "missed branches". "Missed branches" refers to code which branches (e.g. an if statement which has multiple paths, or a switch statement). If we click on a method, we can see the code and what branches are covered and not covered:

image
  • Green lines: Code that was run
  • Red lines: Code that was not run
  • Yellow lines: Some code that was run, but had paths and some of the paths were not run

Loading code coverage test reports in your IDE (Eclipse)

In Eclipse, you can import code coverage test reports and view them directly in your IDE.

I believe the instructions to do this in IntelliJ is as simple as going to Run > Show Code Coverage Data. For more info about code coverage in IntelliJ, see Coverage in the Coverage tool window

JaCoCo generates a jacoco.exec file in commandapi-platforms/commandapi-bukkit/commandapi-bukkit-test/commandapi-bukkit-test-tests/target/jacoco.exec. In Eclipse, you can import this by:

  • Go to Window > Show View > Other...
  • Under Java, select Coverage. This will open the Coverage view
  • Right click in the coverage view and select Import Session...
  • Select the jacoco.exec file from the path above

You now have code coverage reports in your IDE:

image

Tip

To disable code coverage highlighting, press the grey "X" symbol ("Remove Active Session")

Sidebar

Clone this wiki locally