Never debug a test again: Detailed failure reports and hassle free assertions for Java tests - Power Asserts for Java
Clone or download
Latest commit 1383307 Jan 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs Update release_guide.md Jan 7, 2019
scott-examples Bump version Jan 7, 2019
scott-gradle-plugin Bump version Jan 7, 2019
scott-maven-plugin Bump version Jan 7, 2019
scott-tests Bump version Jan 7, 2019
scott Bump version Jan 7, 2019
.gitignore Add gradle files to git ignore Dec 9, 2018
CONTRIBUTING.md Update CONTRIBUTING.md Feb 13, 2018
LICENSE Update license Oct 29, 2016
pom.xml Bump version Jan 7, 2019
readme.md Update readme.md Jan 16, 2019

readme.md

License Maven Central

Scott Test Reporter

Scott provides detailed failure messages for tests written in Java, without the use of complex assertion libraries to aid developers in rapid development, troubleshooting and debugging of tests. All information is presented on the source code of the test method as comments.

test_1(hu.advancedweb.example.CounterTest)  Time elapsed: 0.005 sec  <<< FAILURE!
java.lang.AssertionError: 

   9|      @Test
  10|      public void test_1() {
  11|          Counter counter = new Counter();  // counter=Counter [state=0]
  12|          
  13|          counter.increase();  // counter=Counter [state=1]
  14|          counter.increase();  // counter=Counter [state=2]
  15|          
  16|          int state = counter.get();  // state=2
  17|          
  18|*         assertEquals(state, 3);  // AssertionError: expected:<2> but was:<3>
  19|      }

        at hu.advancedweb.example.CounterTest.test_1(CounterTest.java:18)

It automatically tracks the internal state of the tests to provide important details for a failing scenario, favoring simple assertions expressed mostly in plain Java over the extensive use of test libraries, such as Hamcrest or AssertJ - although Scott plays nicely with other testing tools and frameworks.

Scott does not intend to be a testing framework, nor does it provide an API to use in the tests. Instead, it aims to be a small tool that can be dropped into a project to do its job automatically, so you can worry much less about expressing assertions, and still have meaningful failure messages.

Scott: All systems automated and ready. A chimpanzee and two trainees could run her.

Kirk: Thank you, Mr. Scott. I'll try not to take that personally.

Supports JUnit 4, JUnit 5, and Cucumber Java, on Java 7, Java 8, and Java 11.

How to use

After including Scott to the build flow, it automatically creates the detailed failure messages for failing tests. All you have to do is to write tests in Java with simple assertions or using your favorite testing library and run them as you would do normally. Scott will do its magic behind the scenes.

Gradle

Add hu.advanceweb.scott-gradle-plugin to your build.gradle:

plugins {
  id "hu.advanceweb.scott-gradle-plugin" version "3.4.1"
}

Example projects:

Maven

Add the following to your pom.xml:

<build>
	<plugins>
		<!-- Add the Scott Plugin. -->
		<plugin>
			<groupId>hu.advancedweb</groupId>
			<artifactId>scott-maven-plugin</artifactId>
			<version>3.4.1</version>
			<executions>
				<execution>
					<goals>
						<goal>prepare-agent</goal>
					</goals>
				</execution>
			</executions>
		</plugin>
	</plugins>
</build>
<dependencies>
	<!-- Add Scott as a dependency -->
	<dependency>
		<groupId>hu.advancedweb</groupId>
		<artifactId>scott</artifactId>
		<version>3.4.1</version>
		<scope>test</scope>
	</dependency>
</dependencies>

Example projects:

Cucumber

Scott for Cucumber tracks whole scenarios, and in case of a failure it prints the details of every step involved.

This feature provides valuable information if a test fails in a CI environment, as it can make it much easier to reproduce and fix browser-based tests, especially for flaky tests.

HTML

Example projects with Cucumber tests:

Wire it up manually

If you can't use the Gradle or Maven Plugin for some reason, you can do the necessary steps manually.

Configuration

In case you are not satisfied with the default tracking behavior, the Scott Maven Plugin and Gradle Plugin provides configuration options to fine-tune its behaviour.

Features

Consider this failing test case:

@Test
public void myTest() {
	Integer[] myArray = new Integer[] { 1, 4, 2, 4 };
	List<Integer> myList = Arrays.asList(myArray);

	Set<Integer> mySet = new HashSet<>(myList);
	mySet.remove(4);

	assertTrue(mySet.contains(4));
}

Normally it just produces an assertion error without a meaningful message. But with Scott, it shows additional information:

myTest(hu.advancedweb.example.ListTest) FAILED!
  22|      @Test
  23|      public void myTest() {
  24|          Integer[] myArray = new Integer[] { 1, 4, 2, 4 };  // myArray=[1, 4, 2, 4]
  25|          List<Integer> myList = Arrays.asList(myArray);  // myList=[1, 4, 2, 4]
  26|
  27|          Set<Integer> mySet = new HashSet<>(myList);  // mySet=[1, 2, 4]
  28|          mySet.remove(4);  // mySet=[1, 2]
  29|
  30|*         assertTrue(mySet.contains(4));  // AssertionError
  31|      }

Notice that even without sophisticated assertions the required information is present in the test report.

For every failing test, Scott reports the

  • assignments to local variables,
  • changes made to objects referenced by local variables and fields,
  • input parameters and
  • relevant fields that the test accesses, but does not modify.

All information is nicely visualized on the source code of the test method.

Changelog

See Releases.

Highlights from the latest releases:

Contributing

Contributions are welcome! Please make sure to visit the contribution and development guide for some important notes on how to build and debug Scott. If you are looking for issues that can get you started with the development, see Issues marked with the help-wanted tag.