Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Functional and load testing framework for web applications, written in Python

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.



:author: Benoit Delbosc

:address: bdelbosc _at_

:version: FunkLoad/1.3.0

:revision: $Id$

:Copyright: (C) Copyright 2005 Nuxeo SARL (
    This program is free software; you can redistribute it and/or
    modify it under the terms of the GNU General Public License
    version 2 as published by the Free Software Foundation.
    This program is distributed in the hope that it will be useful, but
    WITHOUT ANY WARRANTY; without even the implied warranty of
    Public License for more details.
    You should have received a copy of the GNU General Public License along
    with this program; if not, write to the Free Software Foundation, Inc.,
    59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

:abstract: This document describes the usage of the FunkLoad_ tool. This tool
    enables to do functional and load testing of web application.

.. sectnum::    :depth: 2

.. contents:: Table of Contents

Introducing FunkLoad

What is FunkLoad ?

FunkLoad_ is a functional and load web tester, written in Python, whose
main use cases are:

* Functional testing of web projects, and thus regression testing as well.

* Performance testing: by loading the web application and monitoring
  your servers it helps you to pinpoint bottlenecks, giving a detailed
  report of performance measurement.

* Load testing tool to expose bugs that do not surface in cursory testing,
  like volume testing or longevity testing.

* Stress testing tool to overwhelm the web application resources and test
  the application recoverability.

* Writing web agents by scripting any web repetitive task, like checking if
  a site is alive.

Main FunkLoad_ features are:

* FunkLoad_ is free software distributed under the `GNU GPL`_.

* Functional test are pure Python scripts using the pyUnit_ framework like
  normal unit test. Python enable complex scenarios to handle real world

* Truly emulates a web browser (single-threaded) using Richard Jones'

  - basic authentication support
  - cookies support
  - referrer support
  - fetching css, javascript and images
  - emulating a browser cache
  - file upload and multipart/form-data submission
  - https support

* Advanced test runner with many command-line options:

  - set the target server url
  - display the fetched page in real time in your browser
  - debug mode
  - green/red color mode

* Turn a functional test into a load test: just by invoking the bench runner
  you can identify scalability and performance problems.

* Detailed bench reports in ReST or HTML (and PDF via ps2pdf)

  - the bench configuration
  - tests, pages, requests stats and charts.
  - the 5 slowest requests.
  - servers cpu usage, load average, memory/swap usage and network traffic
  - an http error summary list

* Easy test customization using a configuration file or command line options.

* Easy test creation using TCPWatch_ as proxy recorder, so you can use your web
  browser and produce a FunkLoad_ test automatically.

* Provides web assertion helpers.

* Provides a funkload.CPSTestCase to ease Zope_ and Nuxeo_ CPS_ testing.

* Easy to install (EasyInstall_) and use, see examples in the demo_ folder.

Where to find FunkLoad ?


* Latest stable package using EasyInstall_::

   sudo easy_install funkload

* EasyInstall_ the latest snapshot ::

   sudo easy_install -f funkload

* Bleeding edge svn sources::

   svn co funkload

See CHANGES.txt_ file for information about distribution contents.


See the INSTALL.txt_ file for requirement and installation.


See the demo_ folder contents and a report_ example.

For package installed with easy_install you need to run ``fl-install-demo``
to extract the demo examples.


Thanks to Frank Cohen's TestMaker_ framework and Richard Jones webunit_

The FunkLoadTestCase

FunkLoadTestCase extends the pyUnit_ unittest.TestCase with browser
capabilities, configuration file helpers and assertions helpers. FunkLoad_
provides also some tools to generate random inputs and communicate with
credential servers.

Browser API


  get(url, params=None, description=None, ok_codes=None)

This emulates a browser http GET link. It will fetch the url, submits
appropriate cookies, follow redirection, register new cookies, load css and
javascript that are not already cached.
This method returns a webunit_ HTTPResponse.


- *url* the url without parameters
- *params* a dico of parameters that going to be append to the url like
- *description* is used on the bench report to describe the user action
- *ok_codes* is a list of http expected code like [200:301] if the http
  response is not in the list `get` will raise a test failure exception,
  if not provided assume that the default list is [200, 301, 302].


  post(url, params=None, description=None, ok_codes=None)

Same interface than the get() but it uses a http post method.
You can upload a file by setting a params like this::

  from webunit.utility import Upload
  params['file_up'] = Upload('/tmp/foo.txt')


  exists(url, params=None, description="Checking existence")

Return True if the http response code is 200, 301 or 302, and return False if
http code is 404 or 503, other codes will raise a test failure exception.


  setBasicAuth(login, password)

Next requests will use the http basic authentication.



Remove basic auth credential set by setBasicAuth.



New in 1.3.0. version.

Set a ``User-Agent`` http header for the next requests, the default browser
behaviour is to use the agent defined in the configuration file under
``[main] user_agent`` or to use the default ``FunkLoad/version``
string. Using this method enable to change the user agent during a test


  addHeader(key, value)

New in 1.3.0. version.

Add an http header for the next requests.



New in 1.3.0. version.

Remove all headers previously added by `addHeader`_ or `setUserAgent`_,
and remove the referer as well.


You can test or bench xmlrpc services using the following API.


  xmlrpc_call(url, method_name, params=None, description=None)

Call the ``method_name`` at ``url`` using xmlrpclib. You can use the
setBasicAuth_ method before to handle the http basic authentication. Note
that due to xmlrpclib limitation you can not use an http proxy.


- *url* the url of the xmlrpc server
- *method_name* the name of the procedure to call
- *params* a list of parameters to pass to the method
- *description* is used on the bench report to describe the action

Configuration file API

A FunkLoadTestCase class uses a configuration file to setup variable
configuration, like the base server url to be tested, the test description,
credential access, logging files and other test specific parameters. The test
configuration file have the same name of the FunkLoadTestCase with a '.conf'
extension. See documented examples in the demo_ folder (``fl-install-demo``).


  conf_get(section, key, default=_marker)

Return an entry from the configuration file. Note that the entry may be
overriden by a command line option.


- *section* the section in the configuration file.
- *key* the key.
- *default* a default value.


Return an integer.


Return a float.


Return a list, the default separators is a colon ':'.

Assertion helpers API

FunkLoad_ uses the unittest assertion (``assert_``, ``assertEquals``,
``fail``, ...), but provides some methods to check the http response.
After fetching a page you can use the following methods.



Return the last accessed page url taking care of redirects.



Return the <base /> href value of the last accessed page.



Return a list of href anchor url present in the last html response,
filtering href using the ``pattern`` regex if present.



Return the last response content.

The response object

The response returned by a get or post are webunit_ HTTPResponse object


  repsonse = self.get(url)
  print "http response code %s" % response.code
  print "http header location %s" % response.headers['location']



getDOM return a SimpleDOM interface of the fetched html page, see the
webunit_ SimpleDOM api instructions for details.


A FunkLoadTestCase store its results in an xml file (like request and test
result) and put other log information into a text log and/or output to the



Debug log message



Information log message

Lipsum API

To generate dummy document contents you can use the funkload.Lipsum api,
this is a very simple "Lorem ipsum" generator.

You can see some examples by doing::

  python -c "from funkload.Lipsum import main; main()"


  from funkload.Lipsum import Lipsum
  lipsum = Lipsum(vocab=V_ASCII, chars=CHARS, sep=SEP)


- *vocab* a list of word, Lipsum provide 3 lists V_ASCII, V_DIAC, V_8859_15
- *chars* the list of char used to build an identifier
- *sep* some separators used in sentences like coma, question mark ...



Return a random word from the vocabulary.


  lipsum.getUniqWord(length_min=None, length_max=None):

Generate a kind of uniq id.


  lipsum.getSubject(length=5, prefix=None, uniq=False,
                    length_min=None, length_max=None)

Return a subject of length word.


- *length* the number of words in the subject
- *prefix* a prefix to add at the beginning of a the subject
- *uniq* add an uniq identifier in the subject
- *length_min/max* the words length is a random between min and max



Return a sentence with some separators and and a ending point.



Return a paragraph of length sentences.



Return a message with length Paragraphs.


  lipsum.getPhoneNumber(lang="fr", format="medium")

Return a random phone number.


- *lang* can be fr or en_US
- *format* can be short, medium or long



Return a random address.


To communicate with FunkLoad_ services like the credential server, there are
some wrappers in the utils module.


  from funkload.utils import xmlrpc_get_credential
  xmlrpc_get_credential(credential_host, credential_port, group=None)

Return a tuple login, password of a user that belong to group if specified.


List groups name served by the credential server.


List all login/password served by the credential server.

Other TestCases

The ZopeTestCase

This class extends the FunkLoadTestCase providing common Zope_ tasks.


  zopeRestart(zope_url, admin_id, admin_pwd, time_out=600)

Stop and Start the Zope_ server.


- *zope_url* the zope url.
- *admin_id* and *admin_pwd* the zope admin credential.
- *time_out* maximum time to wait until the zope server restart.


  zopePackZodb(zope_url, admin_id, admin_pwd, database="main", days=0)

Pack a zodb database.


- *database* the database to pack.
- *days* removing previous revision that are older than *days* ago


  zopeFlushCache(zope_url, admin_id, admin_pwd, database="main")

Remove all objects from all ZODB in-memory caches.


  zopeAddExternalMethod(parent_url, admin_id, admin_pwd,
                        method_id, module, function, run_it=True)

Add an External method an run it.


This class extends the ZopeTestCase providing common Nuxeo_ CPS_ tasks.


  cpsCreateSite(admin_id, admin_pwd,
                manager_id, manager_password,
                manager_mail, langs=None,
                title=None, description=None,
                interface="portlets", zope_url=None, site_id=None)

Build a new CPS_ site.


- *admin_id* and *admin_pwd* the zope admin credential.
- *manager_id* and *manager_pwd* the cps manager credential.
- *zope_url* the Zope_ server url [*]_.
- *site_id* the CPS_ site id.

.. [*] if the zope_url and site_id is not given we guess it using the


  cpsLogin(login, password)

CPS log in.


Logout the user logged in using cpsLogin.



Create a CPS_ group.



Create a CPS_ group if not present.


  cpsCreateUser(user_id=None, user_pwd=None,
                user_givenName=None, user_sn=None,
                user_email=None, groups=None):

Create a CPS_ users.


Create a CPS_ users if not present.


  cpsSetLocalRole(url, name, role)

Grant role to name in url.


  cpsCreateSection(parent_url, title, description)


  cpsCreateWorkspace(parent_url, title, description)



Create a random document in the parent_url container.



Create a simple news in the parent_url container.



Change the ui locale selection



Return a clean list of document href that matches pattern in the previous
page fetched.



Return the list of url that ends with doc_id, using catalog search.

Test runner

A FunkLoad_ test can be used like a standard unittest using a unittest.main()
and a 'python'.

To ease testing FunkLoad_ come with an advanced test runner to override
the static configuration file.


  fl-run-test [options] file [class.method|class|suite] [...]


                        Run default set of tests.
  fl-run-test MyTestSuite
                        Run suite MyTestSuite.
  fl-run-test MyTestCase.testSomething
                        Run MyTestCase.testSomething.
  fl-run-test MyTestCase
                        Run all 'test*' test methods in MyTestCase.
  fl-run-test MyTestCase -u http://localhost
                        Same against localhost.
  fl-run-test -V
                        Run default set of tests and view in real time each
                        page fetch with firefox.
  fl-run-test -h
                        More options.


  --version               show program's version number and exit
  --help, -h              show this help message and exit
  --quiet, -q             Minimal output.
  --verbose, -v           Verbose output.
  --debug, -d             FunkLoad debug output.
  --url=MAIN_URL, -uMAIN_URL
                          Base URL to bench without ending '/'.
                          Minumum sleep time between request.
                          Maximum sleep time between request.
                          Directory to dump html pages.
  --firefox-view, -V      Real time view using firefox, you must have a running
                          instance of firefox in the same host.
  --no-color              Monochrome output.


The same FunkLaod test can be turned into a load test, just by invoking the
bench runner ``fl-run-bench``.


Here are some definitions used in bench mode:

* CUs: Concurrent Users, which is the number of threads.
* STPS: Average of Successful Tests Per Second during a cycle.
* SPPS: Average of Successfully Page Per Second during a cycle.
* RPS: Average Request Per Second, successfully or not.
* max[STPS|SPPS|RPS]: maximum of STPS|SPPS|RPS for a cycle.


A page is an http get/post request with associated sub requests like
redirects, images or links (css, js files). This is what users see as a
single page.


A test is made with 3 methods: setUp/test_name/tearDown. During the test_name
method each get/post request is called a page.


  [setUp][page 1]    [page 2] ... [page n]   [tearDown]
  ======================================================> time
         <----------------------------------> test method
                 <--> sleeptime_min to sleeptime_max
         <-----> page 1 connection time


A cycle is a load of n concurrents test during a 'duration' period.
Threads are launched every 'startupdelay' seconds, each thread executes
test in a loop.

Once all threads have been started we start to record stats.

Only tests that end during the 'duration' period are taken into account
for the test stats (in the representation below test like [---X are not
take into account).

Only pages and requests that finish during the 'duration' are taken into
account for the request and pages statistic

Before a cycle a setUpCycle method is called, after a cycle a tearDownCycle
method is called, you can use these methods to test differents server
configuration for each cycle.


  |n                   [---test--]   [--------]   [--|---X
  |                    |                             |
  |2            [------|--]   [--------]   [-------] |
  |                    |                             |
  |1          [------X | [--------]   [-------]   [--|--X
  |                    |                             |
  |[setUpCycle]        |                             |    [tearDownCycle]
  ===========================================================> time
                       <------ cycle duration ------->
   <----- staging ----->                             <---- staging ----->
              <-> startupdelay    <---> sleeptime


FunkLoad_ can execute many cycles with different number of CUs, this way you
can find easily the maximum number of users that your application can

Running n cycles with the same CUs is a good way to see how the application
handles a writing test over time.

Running n cycles with the same CUs with a reading test and a setUpCycle that
change the application configuration will help you to find the right tuning.


  cvus = [n1, n2, ...]

  |n2                            __________
  |                             /          \
  |                            /            \
  |n1   _________             /              \
  |    /         \           /                \
  |   /           \         /                  \
  |  /             \       /                    \
   ==================================================> time
        <------->   duration     <-------->
                    <-----> cycle sleep time

Bench runner


  fl-run-bench [options] file class.method


  fl-run-bench MyTestCase.testSomething
                        Bench MyTestCase.testSomething using MyTestCase.conf.
  fl-run-bench -u http://localhost:8080 -c 10:20 -d 30 MyTestCase.testSomething
                        Bench MyTestCase.testSomething on localhost:8080
                        with 2 cycles of 10 and 20 users during 30s.
  fl-run-bench -h
                        More options.


  --version               show program's version number and exit
  --help, -h              show this help message and exit
  --url=MAIN_URL, -uMAIN_URL
                          Base URL to bench.
                          Cycles to bench, this is a list of number of virtual
                          concurrent users, to run a bench with 3 cycles with 5,
                          10 and 20 users use: -c 2:10:20
                          Duration of a cycle in seconds.
                          Minimum sleep time between request.
                          Maximum sleep time between request.
                          Startup delay between thread.
  --no-color              Monochrome output.


Here are few remarks/advices to obtain workable metrics.

* Since it uses significant CPU resources, make sure that performance limits
  are not hit by FunkLoad_ before your server's limit is reached.
  Check this by launching a bench from another host.

* Always use description in post/get/xmlrpc_call, this improves the
  readability of the report.

* A good benching test should not be too long so you can bench faster using
  a short cycle duration.

* The cycle duration should be long enought (~5 times the test duration).

* A benching test must have the same number of page and in the same

* You should use many cycles with the same step interval to produce readable
  charts (1:10:20:30:40:50:60 vs 1:10:100)

* Having a cycle with one user give a usefull reference.

* Use a Makefile to make reproductible bench.

Bench report

To produce an HTML or ReST report you need to invoke the ``fl-build-report``,
you can easily produce PDF report using Firefox 'Print To File' in
PostScript then use the ps2pdf converter.


  fl-build-report [options] xmlfile

``fl-build-report`` analyze a FunkLoad_ bench xml result file and output a


  fl-build-report funkload.xml
                        ReST rendering into stdout.
  fl-build-report --html -o /tmp funkload.xml
                        Build an HTML report in /tmp.
  fl-build-report -h
                        More options.


    --version               show program's version number and exit
    --help, -h              show this help message and exit
    --html, -H              Produce an html report.
    --output-directory=OUTPUT_DIR, -oOUTPUT_DIR
                            Directory to store reports.

 Note that you can preview the report for cycles that have been done while
 the bench is still running by invoking the above command.

Test Recorder

Recording a new FunkLoad test

Starting with FunkLoad_ 1.3.0 you can use ``fl-record`` to record your
navigator activity, this requires the TCPWatch_ python proxy see INSTALL.txt_
for information on how to install TCPWatch_.

1. Start the recorder::

    fl-record basic_navigation

  This will output something like this::

    Hit Ctrl-C to stop recording.
    HTTP proxy listening on :8090
    Recording to directory /tmp/tmpaYDky9_funkload.

2. Setup your browser proxy and play your scenario

  * in Firefox: Edit > Preferencies > General; Connection Settings set
    `localhost:8090` as your HTTP proxy

  * Play your scenario using your navigator

  * Hit Ctrl-C to stop recording::

      # Saving uploaded file: foo.png
      # Saving uploaded file: bar.pdf
      Creating script: ./
      Creating configuration file: ./BasicNavigation.conf.

3. Replay you scenario::

     fl-run-test -dV

  You should see all the steps on your navigator.

4. Implement the dynamic part and assertion

  * Code the dynamic part like getting new url of a created document
  * Add assertion using FunkLoad_ helpers
  * Use a credential server if you want to make a bench with different users
    or simply don't want to hard code your login/password.

Note that ``fl-record`` works fine with multi-part encoded form and file upload
but will failed to record https session.

The fl-record command


    fl-record [options] [test_name]

  fl-record launch a TCPWatch_ proxy and record activities, then output
  a FunkLoad script or generates a FunkLoad unit test if test_name is specified.
  The default proxy port is 8090.

  Note that executable must be accessible from your env.


  fl-record foo_bar
                        Run a proxy and create a FunkLoad test case,
                        generates and FooBar.conf file.
                        To test it:  fl-run-test -dV
  fl-record -p 9090
                        Run a proxy on port 9090, output script to stdout.
  fl-record -i /tmp/tcpwatch
                        Convert a tcpwatch capture into a script.


  --version               show program's version number and exit
  --help, -h              show this help message and exit
  --verbose, -v           Verbose output
  --port=PORT, -pPORT     The proxy port.
  --tcp-watch-input=TCPWATCH_PATH, -iTCPWATCH_PATH
                          Path to an existing tcpwatch capture.

Credential server

If you are writing a bench that requires to be logged with different users
FunkLoad_ provides an xmlrpc credential server to serve login/pwd between the
different threads.

It requires 2 files (like unix /etc/passwd and /etc/group) the password file
have the following format::


The group file format is::

  group1:user1, user2
  # you can split group declaration

Setup a configuration file like in the demo_/cmf folder, then start the
credential server::

  fl-credential-ctl credential.conf start

More options::

  fl-credential-ctl --help

See the funkload-demo/cmf example for a credential configuration file.

Monitor server

If you want to monitor a linux server health during the bench, you have to
run a monitor xmlrpc server on the target server, this require to install
the FunkLoad_ package.

On the server side you need to install the FunkLoad_ tool then launch the
server using a configuration file (example in the demo_/simple folder.)::

  fl-monitor-ctl monitor.conf start

  # more info
  fl-monitor-ctl --help

On the bench host side setup your test configuration like this::

  hosts =

  description = The web server
  port = 8008

Then run the bench, the report will include server stats.

Note that you can monitor multiple hosts and that the monitor is linux


* See the trac tickets:

.. _FunkLoad:
.. _TestMaker:
.. _TCPWatch:
.. _webunit:
.. _pyUnit:
.. _INSTALL.txt: INSTALL.html
.. _CHANGES.txt: CHANGES.html
.. _maxq:
.. _Zope:
.. _Cmf:
.. _Nuxeo:
.. _CPS:
.. _`python cheese shop`:
.. _EasyInstall:
.. _demo:
.. _report:
.. _`GNU GPL`:

Something went wrong with that request. Please try again.