Performance tests, just like unit tests, should be kept apart from the real code. Here's a sample of how this can be done with gradle and JMH
A bit more detailed article can be found in my blog.
Usage:
$ gradle benchmarks
$ java -jar build/libs/jmh-gradle-sample-0.0.1-benchmarks.jar
UPDATE: There's now a specialized jmh plugin for gradle