v3.6.8

Improvements

• New command line arguments: --ignore_repl and --skip_tests_with_repl
• Tests whose names begin with an underscore are never invalidated now
• Added more useful trace logs for debugging issues difficult to reproduce

Fixes

• Fix invalid encoding of non-UTF8 characters in the remote reporter
• Added a workaround for the situation when metadata files are corrupted
• Fix snapshot deletion on arm64
• Fix virtual flash drive plugging on arm64

v3.5.0

New snapshot policy

Until now there were only two options: either create snapshots always for a specified test (default behavior), or never create them for that test. The third option is now available: snapshots for specified test will be created as needed and deleted automatically as soon as the are no longer needed. The desired policy can be specified via new snapshots test attribute. Possible values are always (still default behavior), never and auto. For example:

[snapshots: auto]
test A{} 

no_snapshots attribute is still supported for backward compability purposes.

SCSI support

It's possible now to specify bus type for a VM disk. For example:

disk main: {
  bus: "scsi"
  size: 32Gb
}

On x86 the default option is still IDE. On arm64 SCSI is the only option.

Other changes

Now it's possible to use variables in if conditions.

v3.4.0

New depends_on test attribute

This attribute can be specified to force some test to be executed after another one even if they are from different brunches of the inheritance tree. Usage example:

test A {}
test B {}

[depens_on: A, B]
test C {}

New bug action

This action does not affect test execution in any way, It just promote the bug ID to the report, so it can be attached to JIRA or TFS as a tag.

v3.3.0

Self-tests refactoring

The most simple and basic tests were moved from ci to tests folder. This tests do not require nested virtualization now. This simplifies developing process a lot.

New repl action

When the interpterer encounters repl action, it switches to interactive mode. This mode is useful mostly for writing new tests and debugging.

New test planing algorithm

Now the interpreter uses DFS order for tests execution. This means that the interpreter will not move to another branch in the test inheritance tree until it finishes with the current branch. In practice this leads to fewer number of snapshots being created and fewer number of rollbacks.

v3.2.0

Failure categories

During report generation, in case of test failure, the interpreter reports the error category due to which the test has failed, even if the error occurred on the NN-server.

NN-server protocol versioning

Added a mechanism that allows the NN-server to work with older versions of the interpreter. New versions of the interpreter will not work with older versions of the NN-server.

v3.1.8

Integration with Allure framework

Now it's possible to generate reports in a format that is compatible with Allure framework. History of test runs is also supported.

New feature: "variables"

Sometimes it's may be useful to retreive some information from inside of the guest and use it in visual selectors. For example let's assume we need to know a kernel's version. We can pass this information from the guest to the host via guest additions:

exec bash """
    testo-guest-additions-cli set KERNEL_VERSION `uname -r`
"""

After that we can use this information in wait action:

wait "Kernel version is $<KERNEL_VERSION>"

In this case we call KERNEL_VERSION a variable because (unlike the parameters) its value is unknown in advance. For the same reason the syntax of string interpolation for variables is different from the syntax for parameters.

ARM64 support

Testo-lang interpreter supports both x86 and arm64 architectures. Testo NN-server still supports only x86.

Other changes

• GPU id can be specified in the NN-server config
• Some perfomance improvements

v3.0.0

Testo Framework brand new architecture

There is a new component in Testo Framework: a standalone service called testo-nn-server for processing screenshots and searching text and images on them. Thanks to that innovation, Testo Framework is now much more flexible:

1. Now you can separate the screenshots processing logic and place it on a different machine. You can build up a more suitable for screenshots processing machine (for example, with a powerful GPU) and a more hypervisor-suitable machine (more RAM and disk space) and them run Testo Framework in the distributed fashion.
2. One testo-nn-server can manage several machines (instances) working with virtual machines.
3. You can access testo-nn-server even outside of Testo Framework. You can create your own client application and gain information about objects on screenshots.

Please keep in mind that after upgrading Testo to the version 3.0.0 you will have to adjust all JS selectors in your wait, check and mouse click/move actions. Now all JS-selctors must end with a return statement.

Type Autoswitch

Now it's possible to type text with multi-layout letters in type action.

v2.5.0

Screenshot action

This action allow you to make screenshots from virtual machines' screens on demand

Bridge NICs mode

Now you can configure NICs in your virtual machines into Bridge mode

Shared folders support

New way to transfer files between the Host and virtual machines

Text and img detection quality improved

The accuracy of detecting small icons was improved

New format of --report_folder

Now you can run the testo interpreter multiple times with the same --report_folder parameter. Unfortunately, the new report catalog format is not compatible with the old one. --report_logs and --resport_screenshots parameters are deprecated now (--report_folder always has logs and screenshots).

v2.4.2

Macros to generate tests and test benches

There is a new type of macros in Testo-lang: macros with declarations. These macros can encapsulate declarations of virtual entities and tests.

Parameterizing number of pressings in press actions

Now it's possible to parameterize the number of pressings in press actions.

nocheck optional specifier for copyto actions

Now it's possile to disable semantic checks of files existence on the Host in copyto actions. Thanks to this, you can now run tests with copyto actions even if there's no necessary files present of the Host at the moment of running tests. It is assumed that the files will appear on the Host by the time of the actual copying.

Added experimental support for the Windows 10 operating system and the Hyper-V hypervisor

At the moment, not all the functionality of the Testo Lang is supported for the Hyper-V hypervisor. For example, working with virtual flash drives is not yet available. However, many test scenarios already work quite successfully on the Hyper-V hypervisor. In future releases, we will try to minimize the difference in the implementation of functionality for Hyper-V and other hypervisors.

v2.3.0

Boot menu

You can invoke the boot menu in virtual machines now. You can do it with the combination of actions start; press Esc*30.

USB 3 controllers support

Now by default all the virtual machines support the USB 3.0 contoller. You can switch back to the USB 2 conrtoller with the new qemu_enable_usb3 virtual machine attribute with the false value.

New action (un)plug hostdev usb

With the new (un)plug hostdev usb action you can attach Host-plugged USB devices to virtual machines during tests. The integrity of the plugged USB devices are not guaranteed.