ObserveOn: Change to batch dequeue.

[akarnokd]

Using the same batch technique as in SerializedObserver to dequeue and play values. I had to switch to synchronized because ConcurrentLinkedQueue doesn't support drain.

I run the scheduler perf test with it:

Result : 1913,582 ±(99.9%) 29,210 ns/op
  Statistics: (min, avg, max) = (1905,303, 1913,582, 1922,427), stdev = 7,586
  Confidence interval (99.9%): [1884,373, 1942,792]

Result : 64.277,546 ±(99.9%) 734,360 ns/op
  Statistics: (min, avg, max) = (64034,953, 64277,546, 64519,884), stdev = 190,711
  Confidence interval (99.9%): [63543,186, 65011,906]

Result : 67.999.392,293 ±(99.9%) 4715885,625 ns/op
  Statistics: (min, avg, max) = (66777535,133, 67999392,293, 69525773,000), stdev = 1224700,521
  Confidence interval (99.9%): [63283506,668, 72715277,918]

Master for me was 1863,423 / 119.030,536 / 127.738.858,250

Appears to be a 2x speedup on 1024 and 1M items.

[cloudbees-pull-request-builder]

RxJava-pull-requests #1088 SUCCESS
This pull request looks good

[benjchristensen]

Nice improvement. What do you think of us getting a high performing non-blocking queue implementation in RxJava core?

For example, things similar to http://high-scale-lib.sourceforge.net, http://psy-lob-saw.blogspot.com/2013/04/lock-free-ipc-queue.html and discussions in https://groups.google.com/forum/#!forum/mechanical-sympathy

Everywhere we're currently using java.util.concurrent.* for queues, or creating our own that involve synchronized we have opportunities for improvement.

[akarnokd]

I agree. We need more performing queues and I've been looking at options lately to get a few nanoseconds off of schedulers. My thoughts so far on the subject:

• Bounded vs. unbounded: most examples I saw was using bounded queues but we just can't replace our queues with bounded ones because:
  • we don't know how many producers there are and how many messages to "cache" in general
  • many operators, especially the schedulers may enqueue work items from the same thread and would deadlock if the queue is full.
• Throughput vs latency: it appears that most algorithms will come with a tuning parameter to make one or the other better, but rarely both. In general, it might be possible to get an improvement on both in respect of the current implementation.