A tool for finding assumptions on APIs with under-determined specifications.
Java Groovy
Latest commit eea1b67 Feb 3, 2017 @shiloh00 shiloh00 committed with alexgyori Add Gradle plugin for NonDex (#109)
* Init the migration from Maven to Gradle

* Fix JDK version issue when building with Gradle

* Add checkstyle plugin for Gradle

* Fix compilation issues

* Update JDK version

* Add asm package shadowing

* Add Gradle checkstyle for nondex-instrumentation

* Fix jar task in nondex-instrumentation and make shadowJar work

* Clean up the gradle build script

* Generate jar to nondex-instrumentation resource directory

* Add JVM arguments to test execution

* Add checkstyle to maven-plugin

* Update Travis build

* Update README

* Init Gradle plugin for NonDex

* Add Gradle plugin publish configuration

* Add NonDexHelp task

* Update the gradle build script

* Fix bug that nondex-instrumentation.jar will be replace be out.jar

* Add nondexTest task for non-deterministic test

* Update README

* Add execution id

* Reformat the code in Gradle plugin

* Add tests for Gradle plugin

* Update version and README

* Output more information when exception occures

* Update appveyor

* update readme

* rename GenerateOutputJar

* Rename GenerateOutputJar

* fix import

* Rename GenerateOutputJar again

* Replace command line call with java api call

* Use platform independent pathSeparator

* Use separator instead of pathSeparator

* trying to use gradle in building nondex

* matrix setup maybe?

* retrying again :)

* make mvn run in batch mode.

* customizing the build more

* using the wrapper on travis may fix the issues

* Remove whitespace in readme

* Use pathSeparator

* remove gradle build files in other modules

* Move gradlew into module

* Update travis

* Add back checkstyle links

* Update travis

* Update .travis.yml

Trying to fix the travis build.

* Add back the gradle wrapper jar

* Refactor gradle plugin task

* Update build.gradle

* Update readme

* Update version number in build.gradle

* Minor refactor

* Remove redundant dependency declaration

* Remove outdated test assertion

README.md

Build Status Build status Issue Count Codacy Badge

NonDex is a tool for detecting and debugging wrong assumptions on under-determined Java APIs. An example of such an assumption is when code assumes the order of iterating through the entries in a java.util.HashMap is in a specific, deterministic order, but the specification for java.util.HashMap is under-determined and states that this iteration order is not guaranteed to be in any particular order. Such assumptions can hurt portability for an application when they are moved to other environments with a different Java runtime. NonDex explores different behaviors of under-determined APIs and reports test failures under different explored behaviors; NonDex only explores behaviors that are allowed by the specification and any test failure indicates an assumption on an under-determined Java API. NonDex helps expose such brittle assumptions to the developers early, so they can fix the assumption before it becomes a problem far in the future and more difficult to fix.

Supported APIs:

The list of supported APIs can be found here

Prerequisites:

- Java 8 (Oracle JDK, OpenJDK).
- Surefire present in the POM.

Build (Maven):

mvn install

Use (Maven):

To use NonDex, add the plugin to the plugins section under the build section in your pom:

<project>
  ...
  <build>
    ...
    <plugins>
      ...
      <plugin>
        <groupId>edu.illinois</groupId>
        <artifactId>nondex-maven-plugin</artifactId>
        <version>1.1.1</version>
      </plugin>
    </plugins>
  </build>
</project>

To find if you have flaky tests, run:

mvn nondex:nondex

To debug, run:

mvn nondex:debug

The NonDex Maven plugin also offers additional options; to see them all, run:

mvn nondex:help

Use (Gradle):

To use NonDex in Gradle, add the following content into your build.gradle:

buildscript {
  repositories {
    maven {
      url "https://plugins.gradle.org/m2/"
    }
  }
  dependencies {
    classpath "gradle.plugin.edu.illinois:nondex-gradle-plugin:1.2.1"
  }
}

apply plugin: "edu.illinois.nondex"

To find if you have flaky tests, run:

gradle nondexTest

To get the help information of NonDex Gradle plugin, run:

gradle nondexHelp

Use (Command-line):

After installing, if your application uses the same Java version as you use to build NonDex, run:

root=<path to NonDex root>
instrumentedjar=${root}/nondex-instrumentation/resources/out.jar
# Use the instrumented jar to run your application
commonjar=${root}/nondex-common/target/nondex-common-1.1.1.jar
java -Xbootclasspath/p:${instrumentedjar}:${commonjar} <application>

Optionally, in case your application needs a different Java version than the one you use to build NonDex, after installing, run:

root=<path to NonDex root>
instrumentingjar=${root}/nondex-instrumentation/target/nondex-instrumentation-1.1.1.jar
instrumentedjar=${root}/<unique name of the output jar, such as out.jar>
java -jar ${instrumentingjar} <path to rt.jar> ${instrumentedjar}
# Use the instrumented jar to run your application
commonjar=${root}/nondex-common/target/nondex-common-1.1.1.jar
java -Xbootclasspath/p:${instrumentedjar}:${commonjar} <application>

Output:

If there are flaky tests, the output will report them under the section marked "NonDex SUMMARY:"

The flaky tests are also logged in files called "failure" in the .nondex/ directory. Each execution is identified by an execution ID (also reported in the Maven output), and an execution that has a "failure" file will have that "failure" file in a directory in the .nondex/ directory with the same name as the execution ID.

Output (Debug):

After running debugging, the Maven output reports for each flaky test both the command-line arguments to pass in to reproduce the failure and the path to the file containing the debug results for the flaky test. These files are named "debug", and they contain the name of the debugged test and the stack trace for the single invocation point that when run through NonDex leads to the test failing. If the test cannot be debugged to this single point, the Maven output when indicate it by reporting that the cause cannot be reproduced and may be flaky due to other reasons.