You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Vasily Tarasov edited this page May 13, 2017
·
40 revisions
This page contains the complete and rather dry definition of Filebench Workload Model
Language (WML). If you are looking for a gentle introduction to WML proceed to
the Writing workload models page.
WML contains nearly XXX words which can be grouped in four main categories:
Commands define entities and control the overall run. There are four main
entities in Filebench: 1) processes, 2) threads, 3) flowops
(operations), and 4) variables. Every process consists of one or multiple
threads which execute one or multiple flowops in a loop. Commands, entities,
and flowops can be passed parameters using attributes.
Creates all previously defined files and filesets. This would be
automatically done by run or psrun commands but
sometimes it is handy to separate file creation stage. E.g.,
when Filebench is used only to generate a file system tree or
when one wants to execute system commands after file creation
(e.g., drop file system specific caches).
Syntax
create files
echo
The echo command prints the text to the standard output. The text should be
enclosed in quotes and can contain variables.
Syntax:
echo <message>
Example:
echo "Number of files in workload is $files"
enable
Enables additional Filebench features that are disabled by default. Currently
lathist and multi are the features that can be enabled. lathist tells
Filebench to collect per-flowop latency histograms, while multi enables
enables multinode mode. For multinode mode one has to specify master's hostname
and client's hostname. Multinode feature is under development.
Syntax
enable lathistenable multi master=<hostname>,client=<clientname>
Example
enable lathistenable multi master="master.domain.org",client="slave.domain.org"
eventgen
The eventgen command controls the properties of Filebench internal event
generator. The rate attribute of this command sets the number of events
generated per second. Events are then used by eventlimit,
iopslimit, opslimit, and bwlimit
flowops. If there are no events available at the time of a flowop execution
then the flowop blocks until new events are posted. There is one general pool
of events used by all flowops in all threads.
Syntax:
eventgen [rate = <events_per_second>]
Example:
eventgen rate = 100
run
Starts Filebench run. The command causes Filebench to create all defined
filesets and fork defined processes and threads. Threads are then started to
execute corresponding flowops. When the run completes (either because the time
is over or for other reasons), Filebench prints the statistics of the run, e.g.,
ops/sec.
This command is the last that Filebench reads from the workload file. The
reminder of the file is ignored.
Takes runtime as an arument. If runtime is not specified, default runtime
of 60 seconds is used.
Syntax
run [<runtime>]
Example
run 300
set
XXX: The text below about setting
variable values is outdated. In future, we will use
custom variables insted. Furthermore, it is still
to decide if we
Set searches for the varname supplied as the first argument and if not found
creates a new var of that name. It then supplies the integer, string, or
$varname value on the rest of the line to it.
A workload model can make use of variables, known as ''vars''
from within the model. Vars can be user defined, Filebench
internally defined items. All vars are identified by a string preceded by a
dollar sign "$". User defined vars can be created and assigned
either integer or string values using the ''set'' command.
Internal vars can be accessed by specifying the appropriate name
enclosed in braces.
$user_defined_varname
${stats | rate | date |scriptname | hostname}
Variables can be used to set attributes for files, filesets,
processes, threads, and flowops. Each var can hold either a
boolean, 64 bit unsigned integer, double, or character string,
or be empty.
Regular vars
Regular, user defined vars can be set to a value with a ''set''
command.
set $<user_defined_varname> = [true | false | | | ]
Random vars
Random variables are user defined entities that are defined with
a random distribution which is used to pick a random value to
return with each use. The are created with a ''define randvar''
command, and may have individual parameters set with ''set''
commands. The are used just like regular variables, but return a
different value each time they are accessed. They are
particularly useful with flowops.
set $<custom varname> = cvar(type="cvar-type",paramters=mean)
Custom vars
XXX
set mode
The set mode command is used to put Filebench into various special modes of
operation. It is followed by a subcommand, of which quit is the only one
presently defined. The default is quit timeout, which ends the run when the
runtime specified in a run command has expired or when a positively
evaluated finishon* flowop is encountered.
If the workload is expected to end when it runs out of resources, such as files
to delete, then use either quit alldone to finish the run once all the
threads have quit because of resource exhaustion, or quit firstdone, to quit
as soon as the first thread detects resource exhaustion.
Syntax
set mode quit [ timeout | alldone | firstdone ]
Example
set mode quit alldone
system
Executes quoted UNIX command, waits for the command to finish, and prints its
output to the screen.
Syntax
system "<unixcommand>"
Example
system "echo 3 > /proc/sys/vm/drop/caches"system "ls $dir"
define fileset
Information about a group of related files (e.g., all html files accessed by a
Web-server) is contained in a fileset entity. Fileset entities are specified
using the define fileset command. The define fileset command must provide a
name for the fileset and the path to the directory where the files will
reside. In addition, several optional fileset attributes are accepted.
XXX: Describe that filebench maintains internal file system tree.
For every file (whether it exists or not) there is an entry in filebench.
Entry will be marked exists or not, what the file size is.
This will help to explain how reuse and trustree.
Attributes
name =
Mandatory. The name of a fileset.
Fileset names are later used by flowops to refer to specific filesets.
path =
Mandatory. Path to the directory where the files are created.
The path directive is used in the fileset clause to set the prefix path for the
dataset. It is often set at the head of the script or passed in as a parameter:
entries =
Optional. The number of files in a fileset.
This is the maximum number of files that can ever be created in the fileset.
Not all files in the fileset are created
from the very beginning of the Filebench run.
See prealloc attribute to control the ratio
of initially created files.
Default value is 1,024.
Filesets are typically used to create a group of files, and the entries
attribute is used to set the number of such files. If the entries attribute
is not specified, only one file will be created.
filesize = size =
Optional. The size of every file in a fileset.
Default is 1KiB.
The filesize attribute of a file or fileset specifies the size of file(s)
that will be created. If a fileset and filegamma has been specified as other
than 0, the filesize attribute will actually specify the mean file size, with
the actual size of each file based on the gamma distribution with alpha based
on the filegamma attribute.
The size attribute is used with the define file and define fileset
commands. For define file it sets the size of the file. For define fileset, it
sets the mean size of the files, with the actual size set by the gamma random
distribution specified for the fileset.
prealloc =
prealloc
Optional. Percent of the files in a fileset to preallocate before
the actual Filebench workload starts.
Default value, if prealloc is not specified, is 0.
If prealloc is specified without a number
then the value is 100.
The files defined by the file or fileset entities can either exist as potential
files or actual ones. As a potential one, information about them is kept by the
file or fileset entity, but the do not occupy disk space or exist in a
directory. If they do not exist they can be created later with the creatfile
flowop. When used with a file, the prealloc attribute specifies that the file
should actually exist. When used with a fileset, it specifies the percentage of
files that should actually exist, with a default value of 100%.
reuse
Optional. Reuse the fileset if it exists in the file system.
If a file corresponding to a fileset entry marked as existing
exists in a file sytem, then Filebench
will reuse the file. Depending on the trusttree attribute,
such file will or will not be adjusted to the appropriate
size. Depending on the trusttree attribute,zd
if file does not exist in a filesystem, it will or will not be created.
If reuse is not set and a fileset exists, it will
be completely removed before being recreated from scratch.
Optional. Whether to reuse the file or recreate it if it already exists.
the reuse attribute allows the reuse of existing files or filesets which
have the same name as the specified file or fileset. if the file is too large
it will be truncated, and if it is too small it will be rewritten. a fileset
with a matching name will also be reused, with individual files adjusted to
match their new specified sizes.
trusttree
Optional. Whether to completely trust the existing fileset.
Implies reuse attribute and in addition omits file existance
and size checks.
paralloc
Optional. Allocate files on a file system in parallels using 32 threads.
By default, single thread is used.
Use of this attribute can speed up the pre allocation of files, by creating and
writing them in parallel. However, it only works with files at present, not
filesets.
readonly
Optional. Open all files with O_RDONLY. Applies to newly created files as well.
By default O_RDWR flag is used.
writeonly
Optional. Open all files with O_WRONLY. Applies to newly created files as well.
By default O_RDWR flag is used.
dirwidth
Optional. How many files to create per directory.
The dirwidth attribute of a fileset specifies the average number of entries
in each directory created as part of the fileset. Filebench also uses this in
combination with the total number of files in the fileset to calculate the
required mean depth of the fileset's directory tree. The default is a dirwidth
of 0, which specifies a single level directory containing all the files of the
fileset. Default value is XXX.
dirgamma
Optional. Gamma for gamma distribution for files in dirwidth.
The dirgamma attribute of a fileset specifies the alpha parameter of the
gamma distribution which will be used to decide whether a given subdirectory
contains files or additional subdirectories. If the dirgamma attribute is not
specified, it defaults to 1500. The value can range from 100 to 10000, and the
corresponds to a gamma of 0.1 to 10.
** XXX: Should we remove this? I thought about it and then found that origina? *
Defines a single file. Mandatory attributes of the file are name, path
(directory where the file will reside), and size. Optional attributes are
prealloc (create and fill the file with data), paralloc (allocate in
parallel with other files), and reuse (reuse file if exists).
Attributes
name
Mandatory. The name of the file.
During the definition of processes file names are
used in flowops to refer to a specific files or filesets.
path
Mandatory. Path to the directory where the file is created.
The path directive is used in the fileset clause to set the prefix path for the
dataset. It is often set at the head of the script or passed in as a parameter:
size
Optional. The size of files in the fileset. Dedault is 1KiB (1,024B).
The size attribute is used with the define file and define fileset
commands. For define file it sets the size of the file. For define fileset, it
sets the mean size of the files, with the actual size set by the gamma random
distribution specified for the fileset.
prealloc
Optional. Prealloc or not.
reuse
Optional. Whether to reuse the file or recreate it if it already exists.
the reuse attribute allows the reuse of existing files or filesets which
have the same name as the specified file or fileset. if the file is too large
it will be truncated, and if it is too small it will be rewritten. a fileset
with a matching name will also be reused, with individual files adjusted to
match their new specified sizes.
trusttree
Optional. Trust that file exist.
paralloc
Optional. Allocate in parallel with other defined files and filesets.
A Filebench process represents and operating system process and
contains one or more threads. In turn, each Filebench thread
represents an operating system thread of control and contains a
collection of flowops. The following example illustrates how to
specify a process, thread and collection of flowops.
Process entities correspond to operating system processes. The
define process command is used to instantiate a given process
entity, which may spawn one or multiple, identical copies of a
process. Each process consists of one or more threads. A
thread, in turn, consists of a set of flowops (operations)
that define what should a thread do.
Threads can also have a region of memory allocated which certain
flowops will then use as buffer space for I/O. This region is
created by setting a value for the memsize attribute. If the
''useism'' attribute is set then IPC shared memory will be used,
otherwise thread local memory will be used.
When process and threads are specified using the define command, the
instances attribute may be used so request multiple copies of the defined
process or threads. If used with the process definition, the requested number
of operating system processes will be created, each with its own copy of the
threads and flowops included in the definition. Similarly, if used with a
thread definition, the requested number of threads will be created for each
operating system process, each with its own copy of the specified flowops. If
the instance flowop is not included, then a single instance of the process or
thread is created.
**Process attributes**
* `name = `
Mandatory. Name of the process.
* `instances`
Optional. How many instances of this process to spawn.
Default is 1.
the requested number
of operating system processes will be created, each with its own copy of the
threads and flowops included in the definition.
If
the `instance` flowop is not included, then a single instance of the
process is created.
* nice
Optional. Increase nice. Default is not to change the nice.
The `nice` attribute allows you to lower the priority of the process (or set of
processes if multiple instances are requested) below that of what it otherwise
would be. Note that all processes are automatically set to a lower priority
than the master process controlling the run. But if you want a particular
process to be at a lower priority than the others, specify `nice` with some
integer value to accomplish this.
**Thread attributes**
* `name = `
Mandatory. Name of the process.
* `memsize = `
Mandatory. Amount of memory this thread will allocate
in the beginning. The thread will zero-out this region
before the run and will allocate buffers for read and write
flowops from this region. Default is XXXmb.
This parameter in the thread clause sets the size
of the private memory segment of the thread.
* `instances = `
Optional. How many instances of this process to spawn.
Default is 1.
requested number of threads will be created for each
operating system process, each with its own copy of the specified flowops. If
the `instance` flowop is not included, then a single instance of the
thread is created.
* `useism`
Optional. Use shared memory instead of per-thread memory
for all threads.
The `useism` attribute tells the thread to use shared memory for its thread memory region.
**Syntax**
> ``` define process name=<process name>[,instances=<number of instances>][,nice=<additional niceness>][,useism]
{
thread name =<thread name>,memsize=<memory size>[,instances=<number of instances>][,nice=<additional niceness>] {
}
}
list
Depending on the argument, the list command either prints
information about all defined files or flowops.
Syntax
list fileset|flowop
Examples *
list fileset
list flowop
debug
Sets the verbosity of debugging level. Allowed values are from 0
to 10. The higher the value the more debugging information is
printed. By default this is set to 2 at the startup.
Syntax
debug <debug_level>
Example
debug 4
quit
Ends the Filebench execution. Can be used to terminate WML
parsing in the middle of a WML script.
Syntax
quit
sleep
Causes Filebench to sleep for the number of seconds supplied by
the argument. Notice, this is not a flowop executed by workers
but a command that Filebench's master process executes.
Syntax
sleep
Example
sleep 10
version
Prints Filebench version.
Syntax
version
Flowops
The flowop clause determines what a thread actually does. The
operations and their syntax is covered in this section. The
flowop actions can be divided up into Basic I/O, Asynchronous
I/O, Synchronization constructs and Misc other operations.
Some of the flowop syntax is common to all flowops. They are
defined with the flowop keyword within a thread definition.
Immediately following the flowop keyword is the name of the
particular operation that is to be performed. Following that are
a list of attributes, of which the name and iters attributes
are common to all flowops. The name attribute is mandatory and
provides the particular instance of the flowop with a name by
which it can be referenced elsewhere and in the results. The
name must be globally unique. The iters attribute is optional,
but if specified allows the action specified by the flowop to
happen multiple times each time the flowop is invoked.
File I/O Flowops
Reading and writing to files and filesets. On opening or
creating a file, a file descriptor number can be specified,
which will save the returned file descriptor for later use. Then
operations on the already open file can reference it by its file
descriptor number. Other read and write flowops will implicitly
open a file for use if the filename or fileset name is provided
without a file descriptor number. For filesets, specific files
can be accessed by passing a file index number to the flowop,
which can be obtained from a random variable to provide random
file accesses. Otherwise files will be accessed round robin
style.
read
Emulate posix read or pread. The flowop must include either
filesetname or filename attribute. If a filesetname
attribute is supplied, the read operation will be done to a file
from the fileset. Otherwise it will read the file specified by
the filename attribute. If a fileset is specified along with
an fd attribute, than the referenced file will be read.
Otherwise the default fd=0 will be used or, if the opennext
attribute is set, the flowop will pick the next file in sequence
to read. [XXX: this requires more explanation]. If the file is
not already open, this flowop will open it (using, if supplied,
the directio and dsync attributes as described for the
openfile flowop).
The actual read is done to a random offset in the threadflow's
thread memory, with a size set by the iosize attribute and at
a random disk offset within the working set size if the random
attribute is set, or at the next sequential location. The
workingset attribute specifies the the offset range for use in
choosing the random disk offset [XXX: from the beginning or
end?]
Attributes
name =
Mandatory. Name
Names must be globally unique within an entity
type. Thus if you have two processes, each of which has a read type flowop, you
must make sure that both read flowops have unique names, such as name=read1
name=read2.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
opennext
The opennext attribute is used with I/O flowops to indicate that the flowop
should open a different file with each invocation. XXX there is something
funky with this in the code.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
random
Specify that a random location within the file be picked for access. Without
this attribute, the next sequential file blocks will be read or written.
workingset =
The workingset attribute is used by some I/O flowops to specify a maximum
byte range of the file that will actually be read from or written to. This can
be less than the actual file size, or for writes may also be larger than the
current size, where it serves to set the maximum size the file can grow to.
Emulate a read of a whole file. The file from the supplied
fileset filesetname that is referenced by the fd attribute
(if it is supplied) or by the default fd=0. If fil descriptor
(0 or not) is not opened, readwholefile will open the file
before reading it. The readwholefile flowop then reads from
the beginning of the file to the end, using zero or more
iosize reads, followed by a read of whatever remaining, less
than iosize amount, needs to be read. If iosize is not
defined or set to zero, then the file will be read in one read
of filesize bytes (as defined in fileset).
** Attributes **
name
Name of the flowop
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
opennext
The opennext attribute is used with I/O flowops to indicate that the flowop
should open a different file with each invocation. XXX there is something
funky with this in the code.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Emulate a write to a file. The size of the write is specified
by the iosize attribute. If a fileset is specified, it writes
to a file from the fileset referenced by the fd attribute, if it
is supplied, to the default fd=0 file, or to the next file of
the sequence if the opennext attribute is set. [XXX: what
sequnce?]. If a filename attribute is supplied instead, it
will write to the named file. If the file is not already open,
this flowop will open it, using, if provided, the directio and
dsync attributes as described for the openfile flowop. The
flowop's workingset attribute will be used to set the maximum
file size if it is non-zero, otherwise the whole file size will
be used. The actual write is done from a random offset in the
threadflow's thread memory, with a size set by the iosize
attribute and at a random disk offset within the working set
size if the random attribute is set, or at the next
sequential location.
Attributes
name =
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
opennext
The opennext attribute is used with I/O flowops to indicate that the flowop
should open a different file with each invocation. XXX there is something
funky with this in the code.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
random
Specify that a random location within the file be picked for access. Without
this attribute, the next sequential file blocks will be read or written.
workingset =
The workingset attribute is used by some I/O flowops to specify a maximum
byte range of the file that will actually be read from or written to. This can
be less than the actual file size, or for writes may also be larger than the
current size, where it serves to set the maximum size the file can grow to.
Emulates a write of a whole file. The size of the file is
taken from a fileset identified by the srcfd attribute, while
the file used for the write is identified by the fd attribute.
Both default to 0. [XXX: if fd= is set but srcfd is notset,
shouldn't we set srcfd=fd?]. Does multiple writes of iosize
length until full file has been written. If iosize is not
defined or set to zero, then a single write of the size of the
source file is done.
Attributes
name
Mandatory. XXX
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
srcfd =
The srcfd attribute specifies the file descriptor to use as a source of
filesize information when invoking the writewholefile flowop. In the
example below the code is emulating a copy file operation, where the file is
read in then written out to a new file, which, of course, would end up with the
same size as the original.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Emulate a fixed size append to a file. Will append data to a
file chosen from a fileset if one is specified with the
filesetname attribute, or if the fd attribute is non zero
and the attribute's file associated with the file descriptor is
open. If a fileset is specified but the referenced file is not
open, appendfile will open it. If no fileset or non-zero
fd attribute is specified, then a file named by the
''filename'' attribute will be used. If no appropriate file can
be found, Filebench will terminate. While the workingset
attribute is accepted, it is not currently used. Thus the
repeated invocation on the flowop for a given file will cause
the file to grow arbitrarily large. The size of each append is
set by the iosize attribute.
Attribute
name =
Mandatory. XX
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
workingset =
The workingset attribute is used by some I/O flowops to specify a maximum
byte range of the file that will actually be read from or written to. This can
be less than the actual file size, or for writes may also be larger than the
current size, where it serves to set the maximum size the file can grow to.
XXX: Should we remove it and say that a random variable should be
used?
Emulate a random size append to a file. Will append data to a
file chosen from a fileset if one is specified with the
fileset attribute, or if the fd attribute is non zero and
the attribute's file associated with the file descriptor is
open. If a fileset is specified but the referenced file is not
open, appendfile will open it. If no filesetname or non-zero
fd attribute is specified, then a file named by the filename
attribute will be used. If no appropriate file can be found,
Filebench will terminate. Repeated invocation on the flowop for
a given file will cause the file to grow arbitrarily large. A
write to the current end of the file with a random transfer size
of at most ''iosize'' bytes is done.
Attributes
name =
Mandatory.
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the maximum size of the
I/O operation. It defaults to XXX.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Opening, closing and stating files and filesets. On opening or
creating a file, a file descriptor number can be specified.
Then operations on the already open file can reference it by its
file descriptor number. For filesets, specific files can be
opened or created by passing a file index number to the flowop,
which can be obtained from a random variable to provide random
file accesses. Otherwise files will be accessed round robin
style.
createfile
Emulates create of a file. Associates createfile's fd
attribute, if supplied, with the created file's operating system
specific file descriptor so it can be referenced by other file
operations. Selects a file entry from the fileset which file
does not currently exist for the file create operation. Then
performs an open operation on the file with the O_CREATE flag
set to create the file. The file can be created (i.e., opened)
with direct I/O by including the directio attribute and writes
can be forced to behave as defined by synchronized I/O data
integrity constraints by setting the dsync attribute.
Attributes
name =
Mandatory. Name of the flowop.
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Emulates a file open operation. Associates openfile's fd
attribute, if supplied, with the opened file's operating system
specific file descriptor so it can be referenced by other file
operations. However, openfile will fail if the supplied fd
attribute is already associated with an open file. Selects a
file entry from the fileset whose file exists for the file open
operation. Then performs a file open operation on the
filesetentry's associated file. The file can be opened with
direct I/O by including the directio attribute and writes can
be forced to behave as defined by synchronized I/O data
integrity constraints by setting the dsync attribute.
Atributes
name =
Mandatory. Name of the flowop. Must be unique.
directio
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Emulate close of a file. The file referenced by the
fd attribute (if it is supplied) or by default
fd=0, must be open. Simply does a close operation
on the referenced file.
Attributes
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
flowop closefile name= [,fd=]
fsync
Emulates fsync of a file. The file referenced by the
fd attribute (if it is supplied) or by default
fd=0, must be open.
Attributes
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
flowop fsync name=[,fd=]
fsyncset
Emulate fsync of an entire fileset. Does an `fsync``
operation on every open (XX) file of the fileset.
Attributes
name =
Mandatory. Name of the flowop. XXX
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop fsyncset name=,fileset=
statfile
Emulates stat of a file. Picks an arbitrary fileset
entry with an existing file from the fileset specified
by the filesetname attribute, then performs a stat()
operation on it.
Attributes
name =
Mandatory. Name of the flowop. XXX
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Emulates delete of a file. Picks either an arbitrary,
index specified, or file-descriptor-number specified,
filesetentry whose file exists from the fileset
specified by the filesetname attribute, and deletes
it.
Attributes
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
flowop deletefile name=,fileset=[,fd=][,index=]
Directory Flowops
Making, listing and removing directories. The fileset
must have been defined to include empty directory
entries (leafdirs) in addition to or instead of
files. Specific directories can be accessed by passing
a directory index number to the flowop, which can be
obtained from a random variable to provide random
directory accesses. Otherwise directories will be
accessed round robin style.
MakeDir
Emulates mkdir command. Picks either an arbitrary or
index-specified fileset entry for a directory which
does not exist in file system from the fileset
specified by the filesetname attribute, and makes
it.
Attribute
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
XXX: Why FD is needed here?..
flowop makedir name=,fileset=[,fd=][,index=]
ListDir
Emulates ls of a directory. Picks either an
arbitrary or index-specified filesetentry for a
directory which exists in file system from the fileset
specified by the filesetname attribute, and lists
it.
Attributes
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
flowop listdir name=,fileset=[,fd=][,index=]
RemoveDir
Emulates an rmdir. Picks either an arbitrary or
index specified fileset entry for a directory which
exists in file system from the fileset specified by
the filesetname attribute, and removes it.
Attributes
name =
Mandatory. Name of the flowop. XXX
fd =
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
filesetname = filename =
One of this attributes is mandatory.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
XXX: but what happens with fd in the followin iterations?
Syntax
flowop removedir name=,fileset=[,fd=][,index=]
Flowop Asynchronous I/O Operations
aiowrite
Emulates POSIX aiowrite(). Filebench supports asynchronous writes and
implements a mechanism to wait for their completion. An Asynchronous I/O (aio)
element is used to associate the asynchronous write request with its subsequent
completion. An aiowrite flowop will add an aio to the thread's aio list, an
aiowait flowop will wait for half the current list to complete (minimum of
1), removing completed ones from the list.
Attributes
name =
Mandatory. Flowop name.
filesetname = filename =
Mandatory. Fileset to pick a file from. If file is not opened yet,
this flowop will open the file.
The filename attribute specifies the name of a file. It is used with I/O
flowops to specify which particular file to access. The filesetname
attribute specifies the name of a fileset. It is used with I/O flowops to
specify which particular fileset to access.
iosize =
Mandatory?
The iosize attribute is used to specify the size of the
I/O operatio. It defaults to XXX.
random
Mandatory. Offset is picked randomly.
Specify that a random location within the file be picked for access. Without
this attribute, the next sequential file blocks will be read or written.
directio
Optional. Open with DIRECT.
Specifies opening the file in direct, rather than buffered, I/O mode.
Essentially bypasses filesystem caches, so each I/O request results in an
actual I/O to the attached device. Needs to be specified with the flowop that
opens the file, which is often openfile, but can be one of the other I/O
flowops.
dsync
Optional. Open with SYNC.
Specifies the use of synchronous writes, which do not complete until the
attached device has written the data to non volatile storage. Not only does
this disable filesystem write back caching, it also is supposed to prevent
device (such as an attached disk drive) from doing write back caching. This
attribute needs to be specified with all flowops that might open the file, as
the file must be opened as a synchronous file for this attribute to be
effective. While openfile is often used for that purpose, any of the other
flowops will open a file if it is not already open, so they may need the
attribute defined as well.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
opennext
The opennext attribute is used with I/O flowops to indicate that the flowop
should open a different file with each invocation. XXX there is something
funky with this in the code.
workingset =
Optional. Maximum value of the offset to pick withing the file.
The workingset attribute is used by some I/O flowops to specify a maximum
byte range of the file that will actually be read from or written to. This can
be less than the actual file size, or for writes may also be larger than the
current size, where it serves to set the maximum size the file can grow to.
fd =
Optional. File descriptor to use. Default is 0.
The fd parameter to the flowop clause is used to explicitly set the file
descriptor on which the file is opened. This is useful where the script is used
to emulate an application that has a number of files open on different
descriptors or does involved opens/closes using a limited or extended range of
descriptors.
Emulate posix aiowait(). Waits for the completion of half the outstanding
asynchronous I/Os, or a single I/O, which ever is larger. The routine will
return after a sufficient number of asynchronous writes issued by any thread in
the procflow have completed, or a 1 second time-out elapses. All completed I/O
operations are deleted from the thread's list of asynchronous I/Os in progress.
Attributes
name =
Mandatory. Flowop name.
target =
Mandatory. Which aiowrite flowop to wait for.
Blocks the threadflow until woken up by the wakeup flowop.
Attributes
name =
Mandatory. Name of the flowop.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop block name=
Examples
flowop block name=myblock
wakeup
Wakes up one or more blocked target flowops.
Attributes
name =
Mandatory. Name of the flowop.
target =
Name of the target flowop to wake up.
The set of targets consists of all
flowops whose name matches this flowop's target attribute.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop wakeup name=,target=
semblock
Attempts to pass a semaphore and blocks if necessary.
Attributes
name =
Mandatory. Name of the flowop.
value =
Value to decrement before proceeding.
highwater
Maximum value that semaphore can grow up to.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Post to a set of semblock flowops identified by the target attribute.
Attributes
name =
Name of the flowop.
target =
Target semblock flowops to post increment.
value =
Value to post to the semblock.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Consumes CPU cycles and memory bandwidth by looping for value iterations,
while setting the first byte of the thread's memory region to 1 on each
iteration.
Attributes
name =
Mandatory. Name of the flowop.
value =
Mandatory. Number of loops to make.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop hog name=<name>,value=<number-of-mem-ops>
Examples
flowop hog name=myhog,value=1000
delay
Delays for value number of seconds using the user sleep routine.
Attributes
name =
Mandatory. Name of the flowop.
value =
Mandatory. Number of seconds to sleep for.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Completes one invocation per posted event. If events are available, it removes
one and continues to the next flowop, otherwise it blocks until one or more new
events are posted.
Attributes
name =
Mandatory. Name of the flowop.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop eventlimit name=<name>
Examples
flowop eventlimit name=myeventlimit
bwlimit
Blocks the calling thread if the number of bytes of I/O issued exceeds one
megabyte times the number of posted events, thus limiting the average I/O byte
rate to one megabyte times the event rate. To set the event rate see use
eventgen rate command. If a target flowop is specified than the I/O bandwidth
produced by that particular flowop (separately for each thread) sets the limit.
Attributes
name =
Mandatory. Name of the flowop.
target =
Optional. Flowop which I/O is counted. Defaultis is all threads.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Blocks the calling thread if the number of issued I/O operations exceeds the
number of posted events, thus limiting the average I/O operation rate to one
I/O per event. To set the event rate see use eventgen rate command. If a
target flowop is specified than the i/o operations produced by that particular
flowop (separately for each thread) sets the limit.
Attributes
name =
Mandatory. Name of the flowop.
target =
Optional. Flowop which I/O is counted. Defaultis is all threads.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Blocks the calling thread if the number of issued filebench operations exceeds
the number of posted events, thus limiting the average filebench operation rate
to one per event. If a target flowop is specified than the operations
(generally number of times called) produced by that particular flowop
(separately for each thread) sets the limit.
Attributes
name =
Mandatory. Name of the flowop.
target =
Optional. Flowop which I/O is counted. Defaultis is all threads.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Syntax
flowop opslimit name=<name>,[target=<any-flowop]
Examples
flowop opslimit name=myopslimit
finishoncount
Stops the filebench run when the number of any I/O operations specified by
value have been performed. If a target flowop is specified than the
operations count produced by that particular flowop (separately for each
thread) determines when to stop.
Attributes
name =
Mandatory. Name of the flowop.
value=
Mandatory. The number of operations to finish on.
target =
Optional. Flowop which I/O is counted. Defaultis is all threads.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.
Stops the filebench run when the number of bytes of I/O specified by value
have been read and/or written. If a target flowop is specified than the I/O
bandwidth produced by that particular flowop (separately for each thread)
determines when to stop.
Attributes
name =
Mandatory. Name of the flowop.
value=
Mandatory. The number of megabytes to finish on.
target =
Optional. Flowop which I/O is counted. Defaultis is all threads.
iters =
Individual flowops may be executed multiple times each time they are invoked
by setting the iters attribute to the desired number of executions. If not
specified, the flowop will only be executed once each time it is invoked.