Benchmark 9/19 #2

Open
akarnokd opened this Issue Sep 19, 2016 · 2 comments

Projects

None yet

3 participants

@akarnokd
Owner
akarnokd commented Sep 19, 2016 edited

Comparative benchmark

  • RxJava 1.2 (9/17)
  • RxJava 2.0.0-DP0-SNAPSHOT (9/15)
  • Reactor Core 3.0.3-BUILD.SNAPSHOT (9/19)

Environment:

  • i7 4790 stock settings
  • Windows 7 x64
  • Java 8u102
  • JMH 1.13
  • -XX:MaxInlineLevel=20

Green is good, red is not so good.

Subscribe to a simple source

image

Just comparing 1.x and 2.x

image

Remarks:

  • Subscribing to any type in 2.x has lower overhead than 1.x.

Streaming multiple values (synchronous)

image

Remarks:

  • There seems to be some extra overhead in the backpressured concatMapXRange (both 2.x and Reactor).
  • Also flatMapJust has extra overhead for all 2.x types (could be unavoidable atomics).
  • flatMapXRange 1M (which is flatMapJust 1M essentially) suffers the same overhead.
  • 2.x Observable.flatten is not optimized yet and delegates to plain flatMap
  • Otherwise, nice improvements all over the board.

Asynchronous and pipelined streaming

image

Remarks:

  • There seems to be some extra overhead or missing optimization from Flowable async 1.
  • 2.x Observable lack of optimization (1-case) and likely buffer-bloat in longer streams.
  • Generally not bad, but such benchmarks can be hectic (on Windows especially).

Memory usage

image

Remarks:

  • Generally half the memory usage.
  • empty() seems to be occupying too much memory as if there 1M unique empty() instance.
  • Behavior and Unicast seem to occupy double the memory for some reason.

Subject/Processor synchronous streaming

image

Remarks:

  • Async, Publish and small Replay is definitely faster in 2.x
  • Larger Replay, any Unicast and any Behavior are half-to-quarter slower for some unknown reason.

Blocking for an item (or completion)

image

image

image

Remarks:

  • All seem to better at blockingly retrieve one element, except 2.x Observable for some reason.
  • Reactor is optimized on blocking empty() retrieval

Reactor 3 vs RxJava 2

  • They have the same underlying architecture style.
  • Operator internals are practically the same for most Flowable/Flux types.
  • Reactor delegates some of its Mono operators to Flux operator internals.
  • Reactor has more operators fusion-enabled, RxJava has most important operators fusion-enabled, its Observable has not been extensively optimized though.
  • Reactor uses queue-suppliers, RxJava creates specific queues in operators.
  • Reactor uses atomic field updaters, RxJava extends AtomicXXX classes but otherwise uses AtomicXXX instance fields.
  • Reactor has only an onAssembly hook, RxJava has a subscribe-time hook and indirection on top of onAssembly hook.

Subscribing

image

Remarks:

  • Reactor uses a weaker form of backpressure in just and thus slightly better.

Streaming synchronously

image

Remarks:

  • Most operators are within the +/- 5% error margin.
  • Reactor's flatMap seems to be slightly faster (5-15%)
  • Reactor's flatten seems to be reasonably faster (1.5x to 2x).
  • Reactor's iterable seems to be slightly slower (5-15%)

Async streaming

image

Remarks:

  • The async benchmark is a bit hectic, inconclusive which is better
  • Reactor likes longer pipelined streams but dislikes shorter pipelined streams.

Memory use

image

Remarks:

  • Inconclusive, Reactor has more overhead in places and less in others.
  • Reactor's processors are more compacted but leak API.
  • Reactor has some drastic Replay overhead.

Processor overhead

image

Remarks:

  • Reactor is faster in general but slightly slower in large Replay and Unicast

Blocking

image

Remarks:

  • Reactor is optimized for blocking on an empty()

image

Remarks:

  • Blocking for the first element is the same.

image

Remarks:

  • Blocking for the last element is slower in Reactor.

Conclusion

  • RxJava 2.x is generally better in terms of performance and memory utilization than 1.x; most lower performing components can be optimized further.
  • Reactor is generally 10-50% better in a Java 8 environment due to more inlined nature and even fewer allocations than RxJava 2.
@smaldini
Contributor

There's room for improvement in reactor I think given some latest design improvements but this is a massively complete report, congrats @akarnokd !

@daschl
daschl commented Sep 19, 2016

great info, that makes me even more wanting to move the couchbase SDK over to 2.x at some point, but still not sure how to figure out a smooth migration path :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment