This repository has been archived by the owner on Apr 5, 2022. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 218
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
XD-1407 Add sample README and helper script to test the reactor-tcp s…
…ource module.
- Loading branch information
1 parent
85fffc7
commit 171516b
Showing
3 changed files
with
61 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -18,3 +18,4 @@ dump.rdb | |
tomcat.* | ||
activemq-data | ||
spring-shell.log | ||
.testdata |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Reactor TCP Spring XD sample | ||
|
||
This is a sample application demonstrating the use of [Reactor](https://github.com/reactor/reactor) in *Spring XD*. | ||
|
||
## Requirements | ||
|
||
To run this sample you'll need: | ||
|
||
* Spring XD ([Instructions](https://github.com/SpringSource/spring-xd/wiki/Getting-Started)) | ||
* Ruby | ||
* `sendfile` ruby Gem (`gem install sendfile`) | ||
|
||
## Create a Stream | ||
|
||
You'll need to create a `reactor-tcp` source in *Spring XD* using the following example as a template: | ||
|
||
stream create --name reactortcp --definition "reactor-tcp --port=3000 | throughput-sampler" | ||
|
||
You can change the port to which the source is bound by adjusting the `--port` parameter. The default is 3000 and that's what the Ruby script included with this sample will try to connect to, but you can easily change that, of course. | ||
|
||
## Throughput Sampling | ||
|
||
The `throughput-sampler` sink is a simple NOOP sink that looks for a "start" message which, by default is the String `START`. It will count the messages it receives after that point. When it receives an "end" message, which defaults to the not-surprising String value `END`, it will calculate the throughput for the messages it has seen to that point and log that value to the XD log. It's a simple way to measure overall throughput at a fairly coarse level. | ||
|
||
## Send the Data | ||
|
||
We've found in microbenchmarking network servers that it's very difficult to create a simple client that streams test data fast enough to not interfere with the benchmark. Even using a tool like `netcat`, which you would assume would be about the fastest way to send data to a server, is surprisingly inefficient. These inefficiencies add up when sending many thousands of messages. | ||
|
||
The fastest way we've found to send a lot of data to a server very quickly and efficiently (in the hopes you can achieve saturation) is to use the *NIX kernel function "sendfile". How this is done varies based on whether you're using Linux, UNIX, FreeBSD, or Mac OS X. But this test uses the Ruby GEM `sendfile` which has a native extension in it to call this kernel-level function and stream data directly from disk to the network stack, bypassing user space entirely. To do this, it first generates a file of test data that starts with a `START`, contains many thousands of `Hello World!` messages, and is truncated by an `END`. | ||
|
||
Each time your run the `senddata.rb` script, it will generate a file of test data and stream that to the TCP source you created in XD with the above command. The throughput for that run will be logged to the XD log file if you're using the included `throughput-sampler` sink. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
#!/usr/bin/env ruby | ||
|
||
require 'rubygems' | ||
require 'socket' | ||
require 'sendfile' | ||
|
||
PORT = 3000 | ||
MSGCNT = 100000 | ||
|
||
puts "Create an XD stream using the following command:\n\n\tstream create --name reactor --definition \"reactor-tcp --port=#{PORT} | throughput-sampler\"" | ||
|
||
# generate test data | ||
File.open('.testdata', 'w') do |f| | ||
f.puts 'START' | ||
(1..MSGCNT).each do |i| | ||
f.puts 'Hello World!' | ||
end | ||
f.puts 'END' | ||
end | ||
|
||
puts "\nHit [ENTER] when ready to send the data..." | ||
gets | ||
|
||
# use sendfile to efficient send the data | ||
outs = TCPSocket.new 'localhost', PORT | ||
File.open('.testdata', 'r') do |f| | ||
outs.sendfile f | ||
end | ||
outs.close |