Skip to content

Benchmarks

Jump to bottom
Glen K. Peterson edited this page Dec 26, 2015 · 11 revisions

Measured with JMH. A higher score means more operations per second, which is better/faster. Tests were performed on Oracle Java 8 u66 on 64-bit Ubuntu 15.04 running on an Intel E8400 with 8G of DDR3 1300 memory. That processor is worth a whopping $25 on the open market due to it's PassMark score of 2.2k.

#String Performance

This is UncleJim's wheel-house because Strings are immutable objects. For-loops, UncleJim xform()s, and Java 8 streams have essentially the same performance characteristics.

$java -jar target/benchmarks.jar MyBenchmark -f 3 -i 4 -wi 1
...
Benchmark                          Mode  Cnt     Score     Error  Units
MyBenchmark.streamList            thrpt   12   239.531 ±   1.238  ops/s
MyBenchmark.xformList             thrpt   12   234.531 ±   8.211  ops/s
MyBenchmark.forEachLoop           thrpt   12   233.112 ±  11.267  ops/s
MyBenchmark.forILoop              thrpt   12   229.116 ±  34.975  ops/s

The data for this test is in intList which is an ArrayList that contaions ordinal numbers: "1st", "2nd", "3rd", "4th", "5th"...

private static final List<String> intList = new ArrayList<>(NUM_ITEMS);
static {
    System.out.println("Building data...");
    for (int i = 0; i < NUM_ITEMS; i++) {
        intList.add(ordinal(i));
    }
}

We also have two versions of the longest-string function, but they are equivalent:

private static final Function2<String,String,String> longestStr =
        (a, b) -> a.length() > b.length() ? a : b;
private static final BinaryOperator<String> longestStrBinOp =
        (a, b) -> a.length() > b.length() ? a : b;

Here are the 4 equivalent ways of reducing this: Java 8 streams, UncleJim xforms, for-each loop, and C-style for-loop.

// Score: 240 ops/sec streamList
intList.stream().reduce("", longestStrBinOp);

// Score: 235 ops/sec streamList
xform(intList).foldLeft("", longestStr);

// Score: 233 ops/sec forEachLoop
String m = "";
for (String i : intList) {
    m = longestStr.apply(m, i);
}

// Score: 229 ops/sec forILoop
String m = "";
for (int i = 0; i < NUM_ITEMS; i++) {
    String item = intList.get(i);
    m = longestStr.apply(m, item);
}

#Integer Performance

Angelika Langer's blog made Java Generics comprehensible to me and she's been my hero since then. So when she published, "How Fast are Java 8 Streams" I thought I'd compare results. This is not what UncleJim was designed to do. Heck, it's not what Java was designed to do. If you want to do primitive integer array operations really fast, use C or Go. But it's interesting and informative nonetheless.

$java -jar target/benchmarks.jar MyBenchmark -f 3 -i 4 -wi 1
...
Benchmark                          Mode  Cnt     Score     Error  Units
MyBenchmark.forILoopPrimitive     thrpt   12  3598.070 ± 306.088  ops/s
MyBenchmark.forEachLoopPrimitive  thrpt   12  3401.799 ±  47.259  ops/s
MyBenchmark.streamPrimitive       thrpt   12  1930.194 ±  14.555  ops/s
MyBenchmark.xformLambda           thrpt   12   714.523 ±  24.262  ops/s
MyBenchmark.streamLambda          thrpt   12   688.236 ±  12.548  ops/s
MyBenchmark.xformMathMax          thrpt   12   233.774 ±  24.085  ops/s
MyBenchmark.streamMathMax         thrpt   12   231.212 ±   9.812  ops/s

Here we build the test data. There are two sets of data. ints is a primitive array of random numbers. intObjs is an identical ArrayList.

    private static final int NUM_ITEMS = 500000;
    private static final Random prng = new Random();

    private static final int[] ints = new int[NUM_ITEMS];
    private static final List<Integer> intObjs = new ArrayList<>(NUM_ITEMS);
    static {
        System.out.println("Building data...");
        for (int i = 0; i < NUM_ITEMS; i++) {
            ints[i] = prng.nextInt();
            intObjs.add(ints[i]);
        }
    }

Here's the winner:

###forILoopPrimitive: 3,600 ops/sec

int m = Integer.MIN_VALUE;
for (int i = 0; i < NUM_ITEMS; i++) {
    int item = ints[i];
    if (item > m) m = item;
}

This is the oldest kind of Java, iterating through an array of primitives just like a C programmer would in 1972. It mutates in place and directly accesses memory locations. This is probably the closest to how the hardware actually works. But I think most of the speed comes from only updating the variable m very rarely. The "Store" operation must be expensive. If you've done a lot of Java web apps, you probably don't use primitives or arrays. In fact, if most people wrote code like this, we would all still be using C because it's several times faster than the fastest Java.

###forEachLoopPrimitive: 3,400 ops/sec

int m = Integer.MIN_VALUE;
for (int item : ints) {
    if (item > m) m = item;
}

The new forEach loop isn't quite as fast with primitive arrays. It's fast for collections though (see the String tests above). Practically the same code and same speed as the previous.

###streamPrimitive: 1,930 ops/sec

Arrays.stream(ints)
      .reduce(Integer.MIN_VALUE, Math::max);

Java 8 provides 43 different functional interfaces to account for void returns and primitives and this is where it pays off. Personally, I find UncleJim's 3 generic interfaces easier to understand. If I really needed speed for arithmetic operations, I'd code it 1972-style. But you gotta hand it to the Java 8 guys here - this is really fast for functional programming.

###xformLambda: 715 ops/sec

xform(intObjs)
        .foldLeft(Integer.valueOf(Integer.MIN_VALUE), (a, b) -> a > b ? a : b);

Don't run away screaming just yet. Yes, for something better done in C, UncleJim xforms are pretty bad. I mean, Java 8 streams are almost 3x faster. There are several reasons for this:

  • UncleJim doesn't use primitives. In this case, we run into slowness due to comparing Integer objects instead of primitive ints: a.intValue() > b.intValue().
  • UncleJim doesn't update in place. It makes return objects and updates the "max" variable internally on each iteration whether it has changed or not.

There are hundreds of thousands of lines of Java code at my current job, and none of them do this. We don't even use arrays because we heed Josh Bloch's advice and prefer generic collections. If we need a max or min on our business data, we do it in a native SQL database query - not in Java at all.

###streamLambda: 688 ops/sec

intObjs.stream()
      .reduce(Integer.MIN_VALUE, (a, b) -> a > b ? a : b);

When we do the exact same thing in Java 8 streams, with Integer objects in a list and updating the "max" value on each iteration, it's basically the same speed as UncleJim.

###xformMathMax: 234 ops/sec

xform(intObjs)
        .foldLeft(Integer.valueOf(Integer.MIN_VALUE), Math::max);

Ouch - boxing hurts! The problem here is that both the previous maximum Integer and the current Integer being tested need to be converted to primitive ints before being passed to Math.max(int a, int b). That function then returns another primitive int which needs to be converted back to an Integer for the next round. Ouch! Don't do this! It's as slow as the String operations, which defeats the point of using primitives altogether.

###streamMathMax: 231 ops/sec

intObjs.stream()
       .reduce(Integer.MIN_VALUE, Math::max);

Java 8 streams and UncleJim Xforms perform the same in this worst-case scenario. I think that so much time is taken up with boxing, that nothing else really matters.

#Conclusion

If you want to update primitives in place, you don't need UncleJim, or Java 8 Streams. In fact, you might want to program in C or Go instead of JVM languages. If you want error-free, very readable code that leverages generic collections (as Josh Bloch suggests), then there is no performance penalty for using UncleJim.

Clone this wiki locally