A shader script tester for Vulkan
Branch: master
Clone or download
bpeel test-build: Force flushing the memory
The test-build script now sets VKRUNNER_ALWAYS_FLUSH_MEMORY to force
it to call vkFlushMappedMemoryRanges. This will help to test the
memory ranges via the validation layers even on platforms with
coherent memory.

Reviewed-by: Samuel Iglesias Gonsálvez <siglesias@igalia.com>
Latest commit 08280e6 Jan 8, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
android-headers android: Add a dummy vr-vulkan-header.h Nov 19, 2018
vkrunner flush-memory: Add a debug env var to force flushing the memory Feb 18, 2019
.editorconfig Add editorconfig file Nov 8, 2018
.gitignore Convert the build system to CMake Jun 20, 2018
Android.mk Remove vr-feature-offsets Feb 18, 2019
CMakeLists.txt Define VULKAN_HEADER variable to set where external vulkan headers are. Nov 15, 2018
COPYING Add a COPYING file May 22, 2018
README.md README: Add a comment about specifying features from extensions Feb 18, 2019
build-win32.sh Convert the build system to CMake Jun 20, 2018
config.h.in Convert the build system to CMake Jun 20, 2018
precompile-script.py precompile-script.py: add arguments for glslangValidator and spirv-as Oct 16, 2018
test-build.sh test-build: Force flushing the memory Feb 18, 2019
vk-vulkan-header.h.in Define VULKAN_HEADER variable to set where external vulkan headers are. Nov 15, 2018



VkRunner is a Vulkan shader tester based on shader_runner in Piglit. The goal is to make it be able to test scripts as similar to Piglit’s shader_test format as possible.


VkRunner requires the Vulkan headers in order to build. On a Linux system these can be installed via the standard system packages which are libvulkan-dev on Ubuntu and Debian or vulkan-headers on Fedora. On Windows the header can be found by installing LunarG’s VulkanSDK from here.

Additonally VkRunner requires CMake.

If the Vulkan headers are installed in a non-standard location (as will be the case for Windows), you can point CMake to it when configuring the build as follows:

cmake -E env CFLAGS=-Ic:/path/to/vulkan/include cmake .

Otherwise you can just run CMake as below:

cmake .

Next type make to build the program. You will find the VkRunner executable under src/.


VkRunner requires glslangValidator to compile GLSL to SPIR-V. It is invoked on the fly while executing the test. It must either be available in your path or you can set the variable PIGLIT_GLSLANG_VALIDATOR_BINARY to point to it. It can be obtained from here.

[test] section:

The [test] section supports the following commands:

draw rect [ortho] [patch] x y width height

Draws a rectangle at the given normalised coordinates. The vertices will be uploaded at vertex input location 0 as a vec3. Remember that Vulkan’s normalised coordinate system is different from OpenGL’s. If ortho is specified then the coordinates are scaled from the range [0,window size] to [-1,1] to make it easier to specify the positions in pixels. If patch is given then a patch topology will be used with a patch size of four.

draw arrays [indexed] [instanced] topology firstVertex vertexCount [instanceCount]

Calls vkCmdDraw with the given parameters. The vertex data will be sourced from the [vertex data] section. The topology should be one of the values of VkPrimitiveTopology minus the VK_PRIMITIVE_TOPOLOGY prefix. Alternatively it can be a GLenum value as used in Piglit.

If indexed is specified then vkCmdDrawIndexed will be use to draw the primitive instead. The indices will be sourced from the [indices] section. vertexCount will be used as the index count, firstVertex becomes the vertex offset and firstIndex will always be zero.

compute x y z

Dispatch the compute shader with the given parameters.

[relative] probe [rect] (rgb|rgba) (x, y[, width, height]) (r, g, b[, a])

Verifies that a given rectangle matches the given colour. If the command begins with the keyword relative then the coordinates are normalised from 0.0 to 1.0, otherwise they are pixel coordinates. Either way the origin is the top-left corner of the image. If rect is not specified then the width and height are set to 1 pixel. The alpha component of the image can be ignored or not by specifying either rgb or rgba.

probe all (rgb|rgba) r g b [a]

The same as above except that it probes the entire window.

push type offset values

Sets a push constant at the given offset. Note that unlike Piglit, the offset is a byte offset into the push constant buffer rather than a uniform location. The type can be one of int, uint, int8_t, uint8_t, int16_t, uint16_t, int64_t, uint64_t, float, double, vec[234], dvec[234], ivec[234], uvec[234], i8vec[234], u8vec[234], i16vec[234], u16vec[234], i64vec[234], u64vec[234], mat[234]x[234] or dmat[234]x[234]. Multiple values can be specified to upload values to an array of that type. If matrices or arrays are specified they are assumed to have a stride according to the std430 layout rules.

(ubo|ssbo) binding subdata type offset values

Sets a value within a uniform or storage buffer. The first time a value is set within a buffer it will be created with the minimum size needed to contain all of the values set on it via test commands. It will then be bound to the descriptor set at the given binding point. The rest of the arguments are the same as for the uniform command, except that it values are laid out according to the std140 rules for UBOs and std430 for SSBOs. Note that the buffer is just updated by writing into a memory mapped view of it which means that if you do an update, draw call, update and then another draw call both draws will use the values from the second update. This is because the draws are not flushed until the next probe command or the test completes.

(ubo|ssbo) binding size

Sets the size of a uniform or storage buffer. This is optional if there are buffer subdata commands because in that case it will just take the size of the largest offset.

probe ssbo type binding offset comparison values

Probes a value in the storage buffer at binding. The comparison can be one of ==, !=, <, >=, >, <= or ~=. If the type has more than one component then they are compared individually until one of them fails the comparison. ~= is the same with == but ~= allows errors for double or float type numbers while == does not. Allowed errors can be set by the following tolerance command. See examples/tolerance.shader_test for the usage of ~=. Multiple values can be listed to compare an array of values. In that case the buffer is assumed to have std140 layout.

tolerance tolerance0 tolerance1 tolerance2 tolerance3

Sets four tolerances i.e., allowed errors. vecN type values will use first N tolerances among those four. Each column of matMxN type values will also use first N tolerances. float and double type values will use only the first tolerance. Each tolerance value can be an double type real number or percentage e.g., 0.01%. tolerance command can be also used for comparisons of pixels. See examples/tolerance.shader_test for the usage of tolerance command.

tolerance tolerance0

Sets a tolerance i.e., an allowed error. If this command is set, all components of vecN and matMxN type values will use the same tolerance. Each tolerance value can be an double type real number or percentage e.g., 0.01%. See examples/tolerance.shader_test for the usage of tolerance command.

clear color r g b a

Sets the color to use for subsequent clear commands. Defaults to all zeros.

clear depth value

Sets the value to clear the depth buffer to in subsequent clear commands. Defaults to 1.0.

clear stencil value

Sets the value to clear the stencil buffer to in subsequent clear commands. Defaults to 0.


Clears the entire framebuffer to the previously set clear color, depth and stencil values.

patch parameter vertices vertices

Sets the number of control points for tessellation patches in subsequent draw calls. Defaults to 3.

topology, primitiveRestartEnable, patchControlPoints, depthClampEnable, rasterizerDiscardEnable, polygonMode, cullMode, frontFace, depthBiasEnable, depthBiasConstantFactor, depthBiasClamp, depthBiasSlopeFactor, lineWidth, logicOpEnable, logicOp, blendEnable, srcColorBlendFactor, dstColorBlendFactor, colorBlendOp, srcAlphaBlendFactor, dstAlphaBlendFactor, alphaBlendOp, colorWriteMask, depthTestEnable, depthWriteEnable, depthCompareOp, depthBoundsTestEnable, stencilTestEnable, front.failOp, front.passOp, front.depthFailOp, front.compareOp, front.compareMask, front.writeMask, front.reference, back.failOp, back.passOp, back.depthFailOp, back.compareOp, back.compareMask, back.writeMask, back.reference

These properties can be set on a pipeline by specifying their name followed by a value in the test section. This will affect subsequent draw calls. If multiple draw calls are issued with different values for these properties then a separate pipeline will be created for each set of state. See the properties.shader_test example for details.

stage entrypoint name

Sets the entrypoint function to name for the given stage. This will be used for subsequent draw calls or compute dispatches.

uniform type offset values

This is equivalent to push type offset values. It is provided for compatibility with Piglit.

uniform ubo binding type offset values

This is equivalent to ubo binding subdata type offset values. It is provided for compatibility with Piglit.

Take a look in the examples directory for more examples.

[require] section


The [require] section can contain names of members from VkPhysicalDeviceFeatures. These will be searched for when deciding which physical device to open. If no physical device with the corresponding requirements can be found then it will report an error.

In addition to VkPhysicalDeviceFeatures, the name of a feature from any feature struct from an extension that VkRunner is aware of can also be requested. In that case VkRunner will also implicitly require the corresponding device extension. It will also need the VK_KHR_get_physical_device_properties2 instance extension in order to check for the feature. For example, specifying shaderFloat16 in the require section will make it also require the VK_KHR_shader_float16_int8 extension. VkRunner will then enable the feature via the VkPhysicalDeviceFloat16Int8FeaturesKHR struct when creating the device.


Any line that is not a feature and contains entirely alphanumeric and underscore characters is assumed to be a device extension name. This will be checked for when searching for a suitable device and if no device with the extension is found then the test will report that it was skipped. Otherwise the extension will be enabled when creating the device.

framebuffer format

Use this to specify the format of the framebuffer using a format from VkFormat minus the VK_FORMAT prefix.

depthstencil format

If this is specified VkRunner will try to add a depth-stencil attachment to the framebuffer with the given format. Without it no depth-stencil buffer will be created.

fbsize width height

Specify the size of the framebuffer. If not specified it defaults to 250x250.

Shader sections

Shaders can be stored in sections like [vertex shader] just like in shader_runner. Multiple GLSL shaders can be given for a single stage and they will be linked together via glslangValidator.

Alternatively, the disassembly of the SPIR-V source can be provided with a section like [vertex shader spirv]. This will be assembled with spirv-as. If a SPIR-V section is given for a stage there can be no other shaders for that stage.

The vertex shader can also be skipped with an empty section called [vertex shader passthrough]. That will create a simple vertex shader than just copies a vec4 for input location 0 to gl_Position.

[vertex data] section

The [vertex data] section is used to specify vertex attributes and data for use with the draw arrays command. It is similar to Piglit except that integer locations are used instead of names and matrices are specifed by using a location within the matrix rather than having a separate field.

The format consists of a row of column headers followed by any number of rows of data. Each column header has the form ATTRLOC/FORMAT where ATTRLOC is the location of the vertex attribute to be bound to this column and FORMAT is the name of a VkFormat minus the VK_FORMAT prefix.

Alternatively the column header can use something closer the Piglit format like ATTRLOC/GL_TYPE/GLSL_TYPE. GL_TYPE is the GL type of data that follows (“half”, “float”, “double”, “byte”, “ubyte”, “short”, “ushort”, “int” or “uint”), GLSL_TYPE is the GLSL type of the data (“int”, “uint”, “float”, “double”, “ivec”*, “uvec”*, “vec”*, “dvec”*).

The data follows the column headers in space-separated form. “#” can be used for comments, as in shell scripts. See the vertex-data.shader_test file as an example.

[indices] section

The [indices] section just contains a list of indices to use along with the vertices in [vertex data]. It will be used if the indexed option is given to the draw arrays test command.

Long lines

Long lines anywhere in the script can be split into multiple lines by using a backslash to combine them. For example a line to set an array of ints could be split up as follows:

ubo 0 subdata int 0 \
        1 2 3 5 8 13 21 34 55 89 144 233 377 610 \
        987 1597 2584 4181 6765 10946 17711 28657 \
        46368 75025 121393 196418 317811 514229

Command line arguments

usage: vkrunner [OPTION]... SCRIPT...
Runs the shader test script SCRIPT

  -h            Show this help message
  -i IMG        Write the final rendering to IMG as a PPM image
  -d            Show the SPIR-V disassembly
  -D TOK=REPL   Replace occurences of TOK with REPL in the scripts

Precompiling shaders

As an alternative to specifying the shaders in GLSL or SPIR-V assembly, the test scripts can contain a hex dump of the SPIR-V. That way VkRunner does not need to invoke the compiler or assembler to run the script. This can be useful either to speed up the execution of the tests or to run them on hardware where installing the compiler is not practical. VkRunner also includes a Python script to precompile the test scripts to binary. It can be run for example as below:

./precompile-script.py -o compiled-examples examples/*.shader_test
./src/vkrunner compiled-examples/*.shader_test

If glslangValidator and spirv-as are not in the path, you can indicate where the binaries are with the following command line arguments:

./precompile-script.py -o compiled-examples examples/*.shader_test -g PATH_GLSLANG/glslangValidator -s PATH_SPIRV_AS/spirv-as


VkRunner can alternatively be used as a library to integrate it into another test suite. Running make install installs a static library, a header and a pkg-config file to configure it. An example to use it could be as follows:

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

#include <vkrunner/vkrunner.h>

main(int argc, char **argv)
        if (argc != 2) {
                fprintf(stderr, "usage: %s <script>\n", argv[0]);
                return EXIT_FAILURE;

        /* Create a source representing the file */
        struct vr_source *source = vr_source_from_file(argv[1]);

        /* The templating mechanism can be used to replace tokens in
         * the test scripts
        char current_time[64];
        snprintf(current_time, sizeof current_time, "%i", (int) time(NULL));

        /* This executes all of the script and returns a result. The
         * result will be either VR_RESULT_FAIL, VR_RESULT_SKIP or
         * VR_RESULT_PASS.
        struct vr_config *config = vr_config_new();
        struct vr_executor *executor = vr_executor_new(config);
        enum vr_result result = vr_executor_execute(executor, source);


        printf("Test status is: %s\n",

        return result == VR_RESULT_FAIL ? EXIT_FAILURE : EXIT_SUCCESS;

This can by compiled using a command like the following after running make install on VkRunner:

cc -o myrunner myrunner.c $(pkg-config --cflags --libs vkrunner)


VkRunner supports Android NDK build which generates the VkRunner static library and executable for Android.

export ANDROID_NDK=/path/to/your/ndk

cd <vkrunner-dir>
mkdir build && cd build

mkdir libs
mkdir app

$ANDROID_NDK/ndk-build -C ../android_test     \
                      NDK_PROJECT_PATH=.      \
                      NDK_LIBS_OUT=`pwd`/libs \
  • You can see the generated shared library in build/app/local/arm64-v8a/libvkrunner.a.

  • The vkrunner executable which you can run directly on a device will be in build/app/local/arm64-v8a/vkrunner. You may want to first convert any shader scripts to binary SPIR-V using precompile-script.py as described above.