Skip to content
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
543 lines (468 sloc) 21.5 KB

Robot Framework 3.0

Robot Framework 3.0 is a new major release with Python 3 support and a new powerful listener interface that allows modifying executed tests as well as execution results. Also start-up scripts have been enhanced and it is now possible to run tests with new robot command and like python -m robot.

Questions and comments related to the release can be sent to the robotframework-users and possible bugs submitted to the issue tracker.

Robot Framework 3.0 was released on Thursday December 31, 2015.


The easiest way to install or upgrade to the latest version is using pip:

pip install --upgrade robotframework

Alternatively you can download the source distribution from PyPI, extract it, and install it by running:

python install

For more details, including installing Python, Jython and IronPython, see the installation instructions.

The standalone JAR distribution with Jython 2.7 is available at Maven central. Separate Windows installers are not created anymore.

Upgrading from earlier versions

As usual with new major releases, there are some backwards incompatible changes and deprecated features that should be taken into account especially when upgrading larger amount of tests or bigger test infrastructure to Robot Framework 3.0.

Some of the removed features were deprecated only in Robot Framework 2.9. If you encounter lot of errors when upgrading from RF 2.8 or earlier, it might be a good idea to upgrade first to RF 2.9.2 to get deprecation warnings about features that have now be removed altogether.

Most important enhancements

Python 3 support

Robot Framework 3.0 adds support for Python 3 (#1506). More precisely, the currently supported Python versions are 2.6, 2.7, and 3.3 and newer. Installation on Python 3 works exactly as it does for Python 2, and the recommended installation method is with pip:

pip3 install robotframework
python3 -m pip install robotframework

Note that while the standard libraries distributed with Robot Framework do work with Python 3, most other external libraries and tools currently do not.

New listener interface that can modify executed tests and results

New listener API version that receives actual data and result model objects Robot Framework itself uses as arguments has been added (#1208). It is possible to query information from these objects more fluently than when using the old listener API, but more importantly these objects can be modified so that the executed tests and/or created results are affected.

For more information about the new listener API, including interesting usage examples, see the Listener interface section from Robot Framework User Guide.

New robot start-up script

New generic robot start-up script has been introduced to allow running tests like robot tests.robot regardless the Python interpreter (#2216). Old interpreter specific pybot, jybot and ipybot scripts still work as earlier, but the plan is to deprecate and remove them in the future major releases.

The old rebot script has also been made generic and is installed with all Python interpreters. Old jyrebot and ipyrebot scripts work for now.

Possibility to run tests with python -m robot

Earlier it has been possible to run tests with python -m but now also shorter version python -m robot is supported (#2223). Using this approach is especially useful if Robot Framework is used with multiple Python interpreters like, for example, python3 -m robot, /opt/jython27/jython -m robot or ipy -m robot.

A limitation of the python -m robot approach is that it does not work with Python 2.6. The old python -m version can be used instead.

The same enhancement that made python -m robot possible also made it possible to execute the robot installation directory like python path/to/robot. This works the same way as python path/to/robot/ that works also with the earlier versions.

Listeners no longer slow down execution

Listeners used to slow down execution considerably, especially when keyword arguments or return values were objects with a long string presentation (#2241). As part of the speedup fix for this issue, listeners no longer get log messages below the active log level (#2242).

Backwards incompatible changes

No more GUI installers for Windows

Earlier Robot Framework versions have been distributed also as Windows installers, but we have decided not to continue making them in Robot Framework 3.0 (#2218). The ways to install Robot Framework 3.0 are:

  1. Using pip online:

    pip install robotframework
  2. Downloading the source distribution from PyPI and installing it using pip locally:

    pip install robotframework-3.0.tar.gz
  3. Extracting the aforementioned source distribution, navigating to the created directory on the command line, and installing manually:

    python install

OperatingSystem.Start Process and related keywords removed

Keywords related to starting processes in the OperatingSystem library were deprecated in RF 2.9 in favour of better keywords in the Process library. These keywords have been removed in RF 3.0 (#2181) and the aforementioned Process library must be used instead.

Several deprecated commandline options removed

Several command line options were deprecated in RF 2.9 and are now removed in RF 3.0 (#2203). The following table lists the removed options, their replacement, possible short option, and when the replacement was added. Notice that short options have not changed, and they can thus be used if both RF 2.8 and RF 3.0 need to be supported.

Removed Replacement Short option Replacement added
--runfailed --rerunfailed   RF 2.8.4 (#1641)
--rerunmerge --merge   RF 2.8.6 (#1687)
--monitorcolors --consolecolors -C RF 2.9 (#2027)
--monitowidth --consolewidth -W RF 2.9 (#2027)
--monitormarkers --consolemarkers -K RF 2.9 (#2027)

Listener API 1 removed

Old listener API version 1 was deprecated in RF 2.9 in favour of the listener version 2 which was introduced already back in Robot Framework 2.1 (#88). RF 3.0 finally removed the support for the listener API version 1 (#2206). From now on you will always need to specify the API version in your listener with ROBOT_LISTENER_API_VERSION.

Listeners no longer get log message below the active log level

Listeners used to get all log messages to log_message regardless of their log level. This meant that taking a listener into use would cause a considerable amount of TRACE level messages to be generated internally even if the active log level was for example INFO. Now log_message will only be called for messages that would be logged by the current active log level (#2242). If this change breaks some reasonable use case for listeners, we can consider making it possible for listeners to set their own custom log level.

Keyword type passed to listeners has changed

Keyword type passed to listeners was totally messed up with for loops and keyword teardowns. Fixing the problem required changing how types are reported in general. For more information about the new types and the original problem see issue #2248.

Other backwards incompatible changes

  • DeprecatedBuiltIn and DeprecatedOperatingSystem libraries have been removed (#2184).
  • Using same setting multiple times is an error (#2204).
  • Old Meta: Name syntax for specifying test suite metadata has been removed (#2205).
  • Test and keyword timeouts are written to output.xml as a separate <timeout/> element, not as an attribute for the <kw> element (#2092).
  • Executor's local variables can not be used in keyword timeouts (#2092)
  • Console output redirected to a file is written using system encoding, not console encoding (#2197).
  • Aliases from robot.utils.asserts module have been removed (#2202).
  • Changes to internal utility functions and classes (#2200).
  • RunnerFactory Java API has changed (#2090).

Deprecated features

Synonyms for settings

Setting names Document and Suite/Test Pre/Post Condition have been deprecated in favour of Documentation and Suite/Test Setup/Teardown, respectively (#2207). The motivation is to make the overall syntax supported by Robot Framework simpler for users and for external tools. None of examples in the Robot Framework User Guide or in the demo projects have used these deprecated forms, and we assume that they are not widely used. If, however, some of them are in common use, we can still consider removing the deprecation warning and supporting them in the future.

Synonyms for tables

Table names Metadata and User Keyword have been deprecated in favour of Settings and Keywords, respectively (#2208). Similarly as with `deprecated synonyms for settings`_, the motivation is making the syntax simpler, we do not expect them to be used too wildly, and we can consider reverting the deprecation if our expectation is wrong.

Python 2.6 support

Robot Framework 3.0 still supports Python 2.6, but that support can be considered deprecated. The plan is to drop Python 2.6 support in RF 3.1 (#2276).

Other deprecated features

  • Using the WITH NAME syntax case-insensitively is deprecated. Only the all uppercase form will be supported in the future (#2263).
  • Importing libraries with extra spaces in the name like Operating System is deprecated (#2264).
  • Semi public API to register "run keyword variants" has been deprecated in order to be able to redesign it fully in the future (#2265).
  • Using robot.running.TestSuite.(imports|variables|user_keywords) propertys programmatically is deprecated more loudly (#2219).


Many thanks to Jozef Behran for fixing ${TEST_MESSAGE} to reflect current test message (#2188), Michael Walle for Strip String keyword (#2213), and Joong-Hee Lee for adding timeout support for Repeat keyword (#2245).

Full list of fixes and enhancements

ID Type Priority Summary
#1208 enhancement critical New listener API that gets real suite/test objects as arguments and can modify them
#1506 enhancement critical Python 3 support
#2241 bug high Listeners slow down execution, especially when keyword arguments or return values are big
#2216 enhancement high New robot start-up script to replace pybot, jybot and ipybot
#2218 enhancement high No more binary installers for Windows
#2223 enhancement high Support executing tests with python -m robot
#2188 bug medium ${TEST_MESSAGE} is not updated by Set Test Message keyword
#2192 bug medium BuiltIn.Import Resource does not work on standalone jar when no directories in sys.path
#2217 bug medium Error about non-existing variable in keyword return value cannot be caught
#2231 bug medium Parsing massive test case file takes lot of time
#2248 bug medium Keyword type passed to listeners is wrong with for loops and keyword teardowns
#2090 enhancement medium Cleanup RunnerFactory code in Java API
#2092 enhancement medium Possibility to specify keyword timeout using variable provided as argument
#2177 enhancement medium Show critical and non-critical patterns in statistics automatically
#2181 enhancement medium Remove deprecated OperatingSystem.Start Process and related keywords
#2184 enhancement medium Remove DeprecatedBuiltIn and DeprecatedOperatingSystem libraries
#2196 enhancement medium OperatingSystem: Get File and Create File should support native encodings
#2197 enhancement medium Write redirected console output using system encoding, not console encoding
#2198 enhancement medium Process: Allow configuring output encoding
#2213 enhancement medium String: New Strip String keyword
#2229 enhancement medium Screenshot: Support taking screenshot using scrot on Linux
#2238 enhancement medium Officially support imports from sys.path with Import Library/Resource/Variables keywords
#2242 enhancement medium Listeners should not get log messages below the active log level
#2245 enhancement medium BuiltIn: Support also timeout with Repeat Keyword
#2257 enhancement medium Allow using previous arguments in user keyword default values
#2271 enhancement medium Wrap lines from the specified console width when using dotted output
#2275 enhancement medium API docs have general module documentation after submodules
#2279 enhancement medium Enhance public API documentation related to parts exposed to model modifiers and listeners
#2180 bug low Collections: Multiple dictionary keywords fail if keys are unorderable
#2185 bug low Bad error if dynamic or hybrid library returns invalid keyword names
#2243 bug low Using list variable as user keyword argument default value does not work
#2256 bug low Error about non-existing variable in for loop values cannot be caught
#2266 bug low Embedded user keyword arguments are not trace logged
#2267 bug low Dialogs: Closing PASS/FAIL dialog should not be considered same as pressing FAIL
#2268 bug low Convert To Number ignores precision if it is ${0}
#2269 bug low User keyword tags cannot contain list variables
#2121 enhancement low Clarify documentation related to dictionaries originating from YAML variable files
#2200 enhancement low Changes to internal utility functions and classes
#2202 enhancement low Remove aliases from robot.utils.asserts
#2203 enhancement low Remove deprecated command line options
#2204 enhancement low Make it an error if same setting is used multiple times
#2205 enhancement low Remove old Meta: Name syntax for specifying test suite metadata
#2206 enhancement low Remove deprecated listener API version 1
#2207 enhancement low Deprecate Document and Suite/Test Pre/Post Condition synonym settings
#2208 enhancement low Deprecate Metadata and User Keyword table names
#2219 enhancement low Loudly deprecate robot.running.TestSuite.(imports|variables|user_keywords) propertys
#2259 enhancement low Add keyword tags to start/end_keyword listener methods
#2263 enhancement low Deprecate using WITH NAME case-insensitively
#2264 enhancement low Deprecate importing library with extra spaces in name
#2265 enhancement low Deprecate semi-public API to register "run keyword variants" and to disable variable resolving in arguments

Altogether 50 issues. View on issue tracker.

You can’t perform that action at this time.