Skip to content
Browse files

XD-1407 Add sample README and helper script to test the reactor-tcp s…

…ource module.
  • Loading branch information...
1 parent 85fffc7 commit 171516be47aa04ebd381f916c1e2a5ef304d6af8 @jbrisbin jbrisbin committed with garyrussell
Showing with 61 additions and 0 deletions.
  1. +1 −0 .gitignore
  2. +31 −0 reactor-tcp/
  3. +29 −0 reactor-tcp/senddata.rb
1 .gitignore
@@ -18,3 +18,4 @@ dump.rdb
31 reactor-tcp/
@@ -0,0 +1,31 @@
+# Reactor TCP Spring XD sample
+This is a sample application demonstrating the use of [Reactor]( in *Spring XD*.
+## Requirements
+To run this sample you'll need:
+* Spring XD ([Instructions](
+* Ruby
+* `sendfile` ruby Gem (`gem install sendfile`)
+## Create a Stream
+You'll need to create a `reactor-tcp` source in *Spring XD* using the following example as a template:
+ stream create --name reactortcp --definition "reactor-tcp --port=3000 | throughput-sampler"
+You can change the port to which the source is bound by adjusting the `--port` parameter. The default is 3000 and that's what the Ruby script included with this sample will try to connect to, but you can easily change that, of course.
+## Throughput Sampling
+The `throughput-sampler` sink is a simple NOOP sink that looks for a "start" message which, by default is the String `START`. It will count the messages it receives after that point. When it receives an "end" message, which defaults to the not-surprising String value `END`, it will calculate the throughput for the messages it has seen to that point and log that value to the XD log. It's a simple way to measure overall throughput at a fairly coarse level.
+## Send the Data
+We've found in microbenchmarking network servers that it's very difficult to create a simple client that streams test data fast enough to not interfere with the benchmark. Even using a tool like `netcat`, which you would assume would be about the fastest way to send data to a server, is surprisingly inefficient. These inefficiencies add up when sending many thousands of messages.
+The fastest way we've found to send a lot of data to a server very quickly and efficiently (in the hopes you can achieve saturation) is to use the *NIX kernel function "sendfile". How this is done varies based on whether you're using Linux, UNIX, FreeBSD, or Mac OS X. But this test uses the Ruby GEM `sendfile` which has a native extension in it to call this kernel-level function and stream data directly from disk to the network stack, bypassing user space entirely. To do this, it first generates a file of test data that starts with a `START`, contains many thousands of `Hello World!` messages, and is truncated by an `END`.
+Each time your run the `senddata.rb` script, it will generate a file of test data and stream that to the TCP source you created in XD with the above command. The throughput for that run will be logged to the XD log file if you're using the included `throughput-sampler` sink.
29 reactor-tcp/senddata.rb
@@ -0,0 +1,29 @@
+#!/usr/bin/env ruby
+require 'rubygems'
+require 'socket'
+require 'sendfile'
+PORT = 3000
+MSGCNT = 100000
+puts "Create an XD stream using the following command:\n\n\tstream create --name reactor --definition \"reactor-tcp --port=#{PORT} | throughput-sampler\""
+# generate test data'.testdata', 'w') do |f|
+ f.puts 'START'
+ (1..MSGCNT).each do |i|
+ f.puts 'Hello World!'
+ end
+ f.puts 'END'
+puts "\nHit [ENTER] when ready to send the data..."
+# use sendfile to efficient send the data
+outs = 'localhost', PORT'.testdata', 'r') do |f|
+ outs.sendfile f

0 comments on commit 171516b

Please sign in to comment.
Something went wrong with that request. Please try again.