Arcaflow Plugin Uperf

This plugin handles all components of uperf to run the given benchmark profile.

Operation

This plugin contains two parts: the server and the client. Both need to be running for uperf to run benchmarks.

To start the server, specify how long to run it for. It should be at least 5 seconds longer than the client is run.

# yaml-language-server:$schema=uperf_server.input.schema.json

run_duration: 15

To start the client, a profile must be defined in the input.

If using arcaflow to orchistrate uperf, it is recommended that you don't over-complicate the profiles and instead break them down into the tests you want to do, then have arcaflow run them.

A a profile is comprised of groups, which are comprised of transactions, which are comprised of flowops.

For more information, see uperf's documentation and the fully-documented schema for this plugin. All options that uperf profiles support are included in the schema. If you include the schema header in your yaml files and your editor has a compatible yaml extension, you will get suggestions and documentation provided while editing your yaml file.

Example:

# yaml-language-server:$schema=uperf.input.schema.json

name: "netperf"
groups:
  - nthreads: 1
    transactions:
      - iterations: 1
        flowops:
          - type: "accept"
            remotehost: "server"
            protocol: "tcp"
            wndsz: 50
            tcp_nodelay: true
      - duration: "10s"
        flowops:
          - type: "write"
            size: 90
          - type: "read"
            size: 90
      - iterations: 1
        flowops:
          - type: "disconnect"

Running

First, create the yaml files for the client and server inputs. You will pass these into the steps. In the examples below, I use the provided inputs from the input folder.

Without containers

First, create a virtual environment and install the items in the requirements.txt

$ python -m venv .venv
$ source .venv/bin/activate
$ pip install poetry
$ poetry install

Update the input/netperf.yaml:

remotehost: 127.0.0.1

In separate shells on the same or separate machines, you need to run both steps.

The server:

$ python uperf_plugin.py -s uperf_server --file input/server_input.yaml

And the client:

$ python uperf_plugin.py -s uperf --file input/netperf.yaml

With docker-compose or podman-compose

First, build the containers with docker-compose build or podman-compose build

Next, start the containers with docker-compose up or podman-compose up

Lastly, when it's complete, clean up with docker-compose down or podman-compose down

Image Building

You can change this plugin's image version tag in .github/workflows/carpenter.yaml by editing the IMAGE_TAG variable, and pushing that change to the branch designated in that workflow.

Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below

UPerf Run (uperf)

Runs uperf locally

Input

Type:	scope
Root object:	Profile
Properties	comm_port (int) Name: comm_port Description: The master communication port to connect to on the UPerf server. This is used for orchestrating communication with the server (slave). Must match the value specified in the server. Required: No Default (JSON encoded): 20000 Type: int groups (list[reference[ProfileGroup]]) Name: groups Description: A list of groups in the profile. Required: Yes Type: list[reference[ProfileGroup]] List items Type: reference[ProfileGroup] Referenced object: ProfileGroup name (string) Name: name Description: The name you assign to the profile. Does not change the profile's behavior. Required: Yes Type: string
Objects AcceptFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int engine (string) Name: engine Description: The SSL Engine used by OpenSSL Required: No Type: string port (int) Name: port Description: The port used to establish the connection used by the FlowOp. This is not the same as the comm_port. When set to None, UPerf dynamically sets the port. Required: No Type: int protocol (enum[string]) Name: protocol Description: The protocol used to connect to the remote host. Required: No Default (JSON encoded): "tcp" Type: enum[string] Values sctp: SCTP ssl: SSL tcp: TCP udp: UDP vsock: VSOCK rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int remotehost (string) Name: remotehost Description: The remote host server that we need to connect or accept connection from Required: No Default (JSON encoded): "127.0.0.1" Type: string tcp_nodelay (bool) Name: tcp_nodelay Description: Sets the TCP_NODELAY socket option. Required: No Default (JSON encoded): false Type: bool type (string) Required: No Default (JSON encoded): "accept" Type: string wndsz (int) Name: wndsz Description: Size of the socket send and receive buffer in bytes. This parameter is used to set SO_SNDBUF, SO_RCVBUF flags using setsocktopt() Required: No Type: int Units: bytes ConnectFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int engine (string) Name: engine Description: The SSL Engine used by OpenSSL Required: No Type: string port (int) Name: port Description: The port used to establish the connection used by the FlowOp. This is not the same as the comm_port. When set to None, UPerf dynamically sets the port. Required: No Type: int protocol (enum[string]) Name: protocol Description: The protocol used to connect to the remote host. Required: No Default (JSON encoded): "tcp" Type: enum[string] Values sctp: SCTP ssl: SSL tcp: TCP udp: UDP vsock: VSOCK rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int remotehost (string) Name: remotehost Description: The remote host server that we need to connect or accept connection from Required: No Default (JSON encoded): "127.0.0.1" Type: string tcp_nodelay (bool) Name: tcp_nodelay Description: Sets the TCP_NODELAY socket option. Required: No Default (JSON encoded): false Type: bool type (string) Required: No Default (JSON encoded): "connect" Type: string wndsz (int) Name: wndsz Description: Size of the socket send and receive buffer in bytes. This parameter is used to set SO_SNDBUF, SO_RCVBUF flags using setsocktopt() Required: No Type: int Units: bytes DisconnectFlowOp (object) Type: object Properties conn (int) Name: connection id Description: If you know the connection ID, you can specify which one to disconnect. Required: No Type: int count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int type (string) Required: No Default (JSON encoded): "disconnect" Type: string NOPFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int type (string) Required: No Default (JSON encoded): "NOP" Type: string Profile (object) Type: object Properties comm_port (int) Name: comm_port Description: The master communication port to connect to on the UPerf server. This is used for orchestrating communication with the server (slave). Must match the value specified in the server. Required: No Default (JSON encoded): 20000 Type: int groups (list[reference[ProfileGroup]]) Name: groups Description: A list of groups in the profile. Required: Yes Type: list[reference[ProfileGroup]] List items Type: reference[ProfileGroup] Referenced object: ProfileGroup name (string) Name: name Description: The name you assign to the profile. Does not change the profile's behavior. Required: Yes Type: string ProfileGroup (object) Type: object Properties nprocs (int) Name: nprocs Description: The number of processes to run, with each process running a clone of the group Required: No Type: int nthreads (int) Name: nthreads Description: The number of threads to run, with each thread running a clone of the group Required: No Type: int transactions (list[reference[ProfileTransaction]]) Name: transactions Description: A list of transactions in the group Required: Yes Type: list[reference[ProfileTransaction]] List items Type: reference[ProfileTransaction] Referenced object: ProfileTransaction ProfileTransaction (object) Type: object Properties duration (string) Name: duration Description: How long the items in the transaction run. Required: No Type: string flowops (list[one of[string]]) Name: flowops Description: The flowop name for uperf. - connect: Specifies that a connection needs to be opened. - accept: Specifies that a connection needs to be accepted from the remote - disconnect: Disconnects from the existing connection. - read: Data read on the client side (from the server). - write: Data written from the client (to the server). - redv: Receiving a message back from the server to the client. - sendto: A message sent from the client to the server. - sendfile: Uses the sendfile(3EXT) function call to transfer a single file. - sendfilev: Transfers a set of files using the sendfilev(3EXT) interface. Multiple files are randomly picked from all transferrable files (see dir below) and tranferred to the server. - NOP: Does no operation. - think: For period of time with the CPU idling or busy. Required: Yes Type: list[one of[string]] List items Type: one of[string] iterations (int) Name: iterations Description: The number of times the contents of the transaction will run. Required: No Type: int rate (int) Name: rate Description: For limiting the rate. Required: No Type: int ReadFlowOp (object) Type: object Properties canfail (bool) Name: canfail Description: Indicates that a failure for this flowop will not cause uperf to abort. This is espcially useful in UDP where a packet drop does not constitue a fatal error. This can be also be used for example, to test a SYN flood attack (Threads connect() repeatedly ignoring errors). Required: No Default (JSON encoded): false Type: bool conn (int) Name: connection id Description: Every open connection is assigned a connection name. Currently, the name can be any valid integer, however, uperf could take a string in the future. conn identifies the connection to use with this flowop. This connection name is thread private. Required: No Type: int count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int non_blocking (bool) Name: non_blocking Description: Use non-blocking IO. The socket/file descriptor is set the NO_BLOCK flag. Required: No Default (JSON encoded): false Type: bool poll_timeout (string) Name: poll_timeout Description: If this option is set, the thread will first poll for specified duration before trying to carry out the operation. A poll timeout is returned as an error back to uperf. Required: No Type: string randsize_max (int) Name: randsize_max Description: For when a random message size is desired, this sets the max random value, and the size parameter sets the minimum. Unit: bytes. Required: No Type: int Units: bytes rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int rsize (int) Name: rsize Description: Receive size in bytes. For use with asymmetrical messages. For more contect, see description for "size" Required: No Type: int Units: bytes size (int) Name: size Description: Unit: bytes Amount of data that is either read or written. Uperf supports exchange of: - Fixed size messages - Asymmetrical size messages - Random size messages For fixed size messages, the client and all servers used a sized fixed size for receives and transmits. For asymmetrical messages, the slaves use the size specified by the rszize parameter. The master still uses the size parameter. For a random sized message, the a uniformly distributed value between the user specifed min and max is used by the transmitting side, and the receiving side uses the max as the message size. Example: size=64 for 64 bytes per packet Required: No Default (JSON encoded): 64 Type: int Units: bytes type (string) Required: No Default (JSON encoded): "read" Type: string RecvFlowOp (object) Type: object Properties canfail (bool) Name: canfail Description: Indicates that a failure for this flowop will not cause uperf to abort. This is espcially useful in UDP where a packet drop does not constitue a fatal error. This can be also be used for example, to test a SYN flood attack (Threads connect() repeatedly ignoring errors). Required: No Default (JSON encoded): false Type: bool conn (int) Name: connection id Description: Every open connection is assigned a connection name. Currently, the name can be any valid integer, however, uperf could take a string in the future. conn identifies the connection to use with this flowop. This connection name is thread private. Required: No Type: int count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int non_blocking (bool) Name: non_blocking Description: Use non-blocking IO. The socket/file descriptor is set the NO_BLOCK flag. Required: No Default (JSON encoded): false Type: bool poll_timeout (string) Name: poll_timeout Description: If this option is set, the thread will first poll for specified duration before trying to carry out the operation. A poll timeout is returned as an error back to uperf. Required: No Type: string randsize_max (int) Name: randsize_max Description: For when a random message size is desired, this sets the max random value, and the size parameter sets the minimum. Unit: bytes. Required: No Type: int Units: bytes rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int rsize (int) Name: rsize Description: Receive size in bytes. For use with asymmetrical messages. For more contect, see description for "size" Required: No Type: int Units: bytes size (int) Name: size Description: Unit: bytes Amount of data that is either read or written. Uperf supports exchange of: - Fixed size messages - Asymmetrical size messages - Random size messages For fixed size messages, the client and all servers used a sized fixed size for receives and transmits. For asymmetrical messages, the slaves use the size specified by the rszize parameter. The master still uses the size parameter. For a random sized message, the a uniformly distributed value between the user specifed min and max is used by the transmitting side, and the receiving side uses the max as the message size. Example: size=64 for 64 bytes per packet Required: No Default (JSON encoded): 64 Type: int Units: bytes type (string) Required: No Default (JSON encoded): "recv" Type: string SendFileFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int dir (string) Name: dir Description: This parameter identifies the directory from which the files will be transferred. The directory is search recursively to generate a list of all readable files. Example: dir=/space Required: No Default (JSON encoded): "./files" Type: string nfiles (int) Description: This parameter identifies the number of files that will be transferred with each call to sendfilev(3EXT). This is used as the 3rd argument to the sendfilev(3EXT). nfiles is assumed to be 1 for the sendfile flowop function. Example: nfiles=10 Required: No Default (JSON encoded): 1 Type: int rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int size (int) Description: This parameter identifies the chunk size for the transfer. Instead of sending the whole file, uperf will send size sized chunks one at a time. This is used only if nfiles=1. Unit: bytes Required: No Type: int type (string) Required: No Default (JSON encoded): "sendfile" Type: string SendFileVFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int dir (string) Name: dir Description: This parameter identifies the directory from which the files will be transferred. The directory is search recursively to generate a list of all readable files. Example: dir=/space Required: No Default (JSON encoded): "./files" Type: string nfiles (int) Description: This parameter identifies the number of files that will be transferred with each call to sendfilev(3EXT). This is used as the 3rd argument to the sendfilev(3EXT). nfiles is assumed to be 1 for the sendfile flowop function. Example: nfiles=10 Required: No Default (JSON encoded): 1 Type: int rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int size (int) Description: This parameter identifies the chunk size for the transfer. Instead of sending the whole file, uperf will send size sized chunks one at a time. This is used only if nfiles=1. Unit: bytes Required: No Type: int type (string) Required: No Default (JSON encoded): "sendfilev" Type: string SendtoFlowOp (object) Type: object Properties canfail (bool) Name: canfail Description: Indicates that a failure for this flowop will not cause uperf to abort. This is espcially useful in UDP where a packet drop does not constitue a fatal error. This can be also be used for example, to test a SYN flood attack (Threads connect() repeatedly ignoring errors). Required: No Default (JSON encoded): false Type: bool conn (int) Name: connection id Description: Every open connection is assigned a connection name. Currently, the name can be any valid integer, however, uperf could take a string in the future. conn identifies the connection to use with this flowop. This connection name is thread private. Required: No Type: int count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int non_blocking (bool) Name: non_blocking Description: Use non-blocking IO. The socket/file descriptor is set the NO_BLOCK flag. Required: No Default (JSON encoded): false Type: bool poll_timeout (string) Name: poll_timeout Description: If this option is set, the thread will first poll for specified duration before trying to carry out the operation. A poll timeout is returned as an error back to uperf. Required: No Type: string randsize_max (int) Name: randsize_max Description: For when a random message size is desired, this sets the max random value, and the size parameter sets the minimum. Unit: bytes. Required: No Type: int Units: bytes rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int rsize (int) Name: rsize Description: Receive size in bytes. For use with asymmetrical messages. For more contect, see description for "size" Required: No Type: int Units: bytes size (int) Name: size Description: Unit: bytes Amount of data that is either read or written. Uperf supports exchange of: - Fixed size messages - Asymmetrical size messages - Random size messages For fixed size messages, the client and all servers used a sized fixed size for receives and transmits. For asymmetrical messages, the slaves use the size specified by the rszize parameter. The master still uses the size parameter. For a random sized message, the a uniformly distributed value between the user specifed min and max is used by the transmitting side, and the receiving side uses the max as the message size. Example: size=64 for 64 bytes per packet Required: No Default (JSON encoded): 64 Type: int Units: bytes type (string) Required: No Default (JSON encoded): "sendto" Type: string ThinkFlowOp (object) Type: object Properties count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int duration (string) Required: No Type: string rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int think_type (enum[string]) Name: Think Type Description: The type of think type, either idle, whichsleeps the CPU threads, or busy, which uses CPU time. Required: No Default (JSON encoded): "idle" Type: enum[string] Values busy: BUSY idle: IDLE type (string) Required: No Default (JSON encoded): "think" Type: string WriteFlowOp (object) Type: object Properties canfail (bool) Name: canfail Description: Indicates that a failure for this flowop will not cause uperf to abort. This is espcially useful in UDP where a packet drop does not constitue a fatal error. This can be also be used for example, to test a SYN flood attack (Threads connect() repeatedly ignoring errors). Required: No Default (JSON encoded): false Type: bool conn (int) Name: connection id Description: Every open connection is assigned a connection name. Currently, the name can be any valid integer, however, uperf could take a string in the future. conn identifies the connection to use with this flowop. This connection name is thread private. Required: No Type: int count (int) Name: count Description: The number of times this flowop will be executed. Required: No Type: int non_blocking (bool) Name: non_blocking Description: Use non-blocking IO. The socket/file descriptor is set the NO_BLOCK flag. Required: No Default (JSON encoded): false Type: bool poll_timeout (string) Name: poll_timeout Description: If this option is set, the thread will first poll for specified duration before trying to carry out the operation. A poll timeout is returned as an error back to uperf. Required: No Type: string randsize_max (int) Name: randsize_max Description: For when a random message size is desired, this sets the max random value, and the size parameter sets the minimum. Unit: bytes. Required: No Type: int Units: bytes rate (int) Name: rate Description: Experimental: This option causes uperf to execute this flowop at the specified rate for iterations or duration seconds. Required: No Type: int rsize (int) Name: rsize Description: Receive size in bytes. For use with asymmetrical messages. For more contect, see description for "size" Required: No Type: int Units: bytes size (int) Name: size Description: Unit: bytes Amount of data that is either read or written. Uperf supports exchange of: - Fixed size messages - Asymmetrical size messages - Random size messages For fixed size messages, the client and all servers used a sized fixed size for receives and transmits. For asymmetrical messages, the slaves use the size specified by the rszize parameter. The master still uses the size parameter. For a random sized message, the a uniformly distributed value between the user specifed min and max is used by the transmitting side, and the receiving side uses the max as the message size. Example: size=64 for 64 bytes per packet Required: No Default (JSON encoded): 64 Type: int Units: bytes type (string) Required: No Default (JSON encoded): "write" Type: string

Outputs

error

Type:	scope
Root object:	UPerfError
Properties	error (string) Required: Yes Type: string
Objects UPerfError (object) Type: object Properties error (string) Required: Yes Type: string

success

Type:	scope
Root object:	UPerfResults
Properties	profile_name (string) Name: profile_name Description: The name of the profile that was run. Required: Yes Type: string timeseries_data (map[int,map[int,reference[UPerfRawData]]]) Name: timeseries_data Description: Map of the transaction ID to a map of timestamp to UPerfRawData. This result varies by input profile, due to it correlating to the transactions in the profile. It ignores zero initial results. Required: Yes Type: map[int,map[int,reference[UPerfRawData]]] Key type Type: int Value type Type: map[int,reference[UPerfRawData]] Key type Type: int Value type Type: reference[UPerfRawData] Referenced object: UPerfRawData
Objects UPerfRawData (object) Type: object Properties nr_bytes (int) Required: Yes Type: int nr_ops (int) Required: Yes Type: int ns_per_op (int) Required: Yes Type: int UPerfResults (object) Type: object Properties profile_name (string) Name: profile_name Description: The name of the profile that was run. Required: Yes Type: string timeseries_data (map[int,map[int,reference[UPerfRawData]]]) Name: timeseries_data Description: Map of the transaction ID to a map of timestamp to UPerfRawData. This result varies by input profile, due to it correlating to the transactions in the profile. It ignores zero initial results. Required: Yes Type: map[int,map[int,reference[UPerfRawData]]] Key type Type: int Value type Type: map[int,reference[UPerfRawData]] Key type Type: int Value type Type: reference[UPerfRawData] Referenced object: UPerfRawData

UPerf Server (uperf_server)

Runs the passive UPerf server to allow benchmarks between the client and this server

Input

Type:	scope
Root object:	UPerfServerParams
Properties	comm_port (int) Name: comm_port Description: The master communication port to open for this UPerf server. This is used for orchestrating communication between the UPerf server (slave) and the UPerf client (master). Must match the value specified in the client. Required: No Default (JSON encoded): 20000 Type: int run_duration (int) Name: run_duration Description: How long the UPerf server should run before terminating. 0 for indefinite. Required: No Default (JSON encoded): 60 Type: int
Objects UPerfServerParams (object) Type: object Properties comm_port (int) Name: comm_port Description: The master communication port to open for this UPerf server. This is used for orchestrating communication between the UPerf server (slave) and the UPerf client (master). Must match the value specified in the client. Required: No Default (JSON encoded): 20000 Type: int run_duration (int) Name: run_duration Description: How long the UPerf server should run before terminating. 0 for indefinite. Required: No Default (JSON encoded): 60 Type: int

Outputs

error

Type:	scope
Root object:	UPerfServerError
Properties	error (string) Required: Yes Type: string error_code (int) Required: Yes Type: int
Objects UPerfServerError (object) Type: object Properties error (string) Required: Yes Type: string error_code (int) Required: Yes Type: int

success

Type:	scope
Root object:	UPerfServerResults
Properties
Objects UPerfServerResults (object) Type: object Properties