-
Notifications
You must be signed in to change notification settings - Fork 14
/
rds-stress.1
174 lines (172 loc) · 7.29 KB
/
rds-stress.1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
.Dd May 15, 2007
.Dt RDS-STRESS 1
.Os
.Sh NAME
.Nm rds-stress
.Nd send messages between processes over RDS sockets
.Pp
.Sh SYNOPSIS
.Nm rds-stress
.Bk -words
.Op Fl p Ar port_number
.Op Fl r Ar receive_address
.Op Fl s Ar send_address
.Op Fl a Ar ack_bytes
.Op Fl q Ar request_bytes
.Op Fl D Ar rdma_bytes
.Op Fl d Ar queue_depth
.Op Fl t Ar nr_tasks
.Op Fl c
.Op Fl R
.Op Fl V
.Op Fl v
.Sh DESCRIPTION
.Nm rds-stress
sends messages between groups tasks, usually running on seperate
machines.
.Pp
First a passive receiving instance is started.
.Pp
.Dl $ rds-stress
.Pp
Then an active sending instance is started, giving it
the address and port at which it will find a listening
passive receiver. In addition, it is given configuration options which
both instances will use.
.Pp
.Dl $ rds-stress -s recvhost -p 4000 -t 1 -d 1
.Pp
The active sender will parse the options, connect to the passive receiver, and
send the options over this connection. From this point on both instances
exhibit the exact same behaviour.
.Pp
They will create a number of child tasks as specified by the -t option. Once
the children are created the parent sleeps for a second at a time, printing a
summary of statistics at each interval.
.Pp
Each child will open an RDS socket, each binding to a port number in order
after the port number given on the command line. The first child would bind to
port 4001 in our example. Each child sets the send and receive buffers to
exactly fit the number of messages, requests and acks, that will be in flight
as determind by the command line arguments.
.Pp
The children then enter their loop. They will keep a number of sent messages
outstanding as specified by the -d option. When they reach this limit they
will wait to receive acks which will allow them to send again. As they receive
messages from their peers they immediately send acks.
.Pp
Every second, the parent process will display statistics of the ongoing
stress test. The output is described in section OUTPUT below.
.Pp
If the -T option is given, the test will terminate after the specified time,
and a summary is printed.
.Pp
Each child maintains outstanding messages to all other children of the other instance.
They do not send to their siblings.
.Sh OPTIONS
The following options are available for use on the command line:
.Bl -tag -width Ds
.It Fl p Ar port_number
Each parent binds a TCP socket to this port number and their respective
address. They will trade the negotiated options over this socket. Each
child will bind an RDS socket to the range of ports immediately following
this port number, for as many children as there are.
.It Fl s Ar send_address
A connection attempt is made to this address. Once its complete and the
options are sent over it then children will be created and work will proceed.
.It Fl r Ar receive_address
This specifies the address that messages will be sent from. If -s is not
specified then rds-stress waits for a connection on this address before
proceeding.
.Pp
If this option is not given, rds-stress will choose an appropriate address.
The passive process will accept connections on all local interfaces, and
obtain the address once the control connection is established.
The active process will choose a local address based on the interface through
which it connects to the destination address.
.It Fl a Ar ack_bytes
This specifies the size of the ack messages, in bytes. There is a minimum size
which depends on the format of the ack messages, which may change over time.
See section "Message Sizes" below.
.It Fl q Ar request_bytes
This specifies the size of the request messages, in bytes.
It also has a minimum size which may change over time.
See section "Message Sizes" below.
.It Fl D Ar rdma_bytes
RDSv3 is capable of transmitting part of a message via RDMA directly from
application buffer to application buffer. This option enables RDMA support
in rds-stress: request packets include parameters for an RDMA READ or WRITE
operation, which the receiving process executes at the time the ACK packet
is sent.
See section "Message Sizes" below.
.It Fl d Ar queue_depth
Each child will try to maintain this many sent messages outstanding to each
of its peers on the remote address.
.It Fl t Ar nr_tasks
Each parent will create this many children tasks.
.It Fl T Ar seconds
Specify the duration of the test run. After the specified number of seconds,
all processes on both ends of the connection will terminate, and the
active instance will print a summary. By default, rds-stress will keep
on sending and receiving messages.
.It Fl z
This flag can be used in conjunction with -T. It suppresses the ongoing
display of statistics, and prints a summary only.
.It Fl c
This causes rds-stress to create child tasks which just consume CPU cycles.
One task is created for each CPU in the system. First each child observes the
maximum rate at which it can consume cycles. This means that this option
should only be given on an idle system. rds-stress can then calculate the CPU
use of the system by observing the lesser rate at which the children consume
cycles. This option is *not* shared between the active and passive instances.
It must be specified on each rds-stress command line.
.It Fl R
This tells the rds-stress parent process to run with SCHED_RR priority,
giving it precedence over the child processes. This is useful when running
with lots of tasks, where there is a risk of the child processes starving
the parent, and skewing the results.
.It Fl v
With this option enabled, packets are filled with a pattern that is
verified by the receiver. This check can help detect data corruption
occuring under high load.
.El
.Pp
.Ss Message Sizes
Options which set a message size (such as -a) specify a number of bytes
by default. By appending \fBK\fP, \fBM\fP, or \fBG\fP, you can specify the size
in kilobytes, megabytes or gigabytes, respectively. For instance,
the following will run rds-stress with a message and ACK size of 1024
bytes, and an RDMA message size of 1048576 bytes:
.Pp
.Dl rds-stress ... -q 1K -a 1K -D 1M
.Pp
.Pp
.Sh OUTPUT
Each parent outputs columns of statistics at a regular interval:
.Bl -tag -width Ds
.It tsks
The number of child tasks which are running.
.It tx/s
The number of sendmsg() calls that all children are executing, per second.
.It tx+rx K/s
The total number of bytes that are flowing through sendmsg() and recvmsg() for all children.
This includes both request and ack messages.
.It rw+rr K/s
The total number of bytes that are being transferred via RDMA READs and
WRITEs for all children.
.It tx us/c
The average number of microseconds spent in sendmsg() calls.
.It rtt us
The average round trip time for a request and ack message pair. This measures
the total time between when a task sends a request and when it finally receives
the ack for that message. Because it includes the time it takes for the
receiver to wake up, receive the message, and send an ack, it can grow to be
quite large under load.
.It cpu %
This is the percentage of available CPU resources on this machine that are being
consumed since rds-stress started running. It will show -1.00 if -c is not
given. It is calculated based on the amount of CPU resources that CPU soaking
tasks are able to consume. This lets it measure CPU use by the system, say in
interrupt handlers, that task-based CPU accounting does not include.
For this to work rds-stress must be started with -c on an idle system.
.El