Skip to content

Comprehensive testing tool for Vim


Notifications You must be signed in to change notification settings


Folders and files

Last commit message
Last commit date

Latest commit



32 Commits

Repository files navigation

testing.vim is a testing tool for Vim with a design loosely inspired by Go's testing package. It includes support for testing syntax highlighting, benchmarking, and syntax highlighting, linting, and coverage reports (via covimerage).


Annotated example test function (original):

fun! Test_cursor_offset() abort
    " Create a new bufer, and add three lines to it. Make sure the cursor is on
    " the third line.
    call append(0, ['', 'aaa', 'bbb'])
    call cursor(3, 0)

    " Call the function we want to test and ensure we have the correct output.
    let l:out = gopher#internal#cursor_offset()
    call assert_equal(8, l:out)

    " Again with a different parameter.
    let l:out = gopher#internal#cursor_offset(1)
    call assert_equal(':#8', l:out)

    " Write the buffer; it's chdir()'ed to a tmp dir, so it's fine to just write
    " stuff.
    silent w off

    " Ensure filename is added.
    let l:out = gopher#internal#cursor_offset(1)
    call assert_equal(g:test_tmpdir . '/off:#8', l:out)


testing.vim is not a Vim plugin. Clone/download it and run the tvim script to launch it. You can put this directory in your PATH, or use make to install it:

$ make install                     # To /usr/local
$ PREFIX=~/.local make install     # Non-root install.

The test of the documentation will assume tvim is in PATH.

Run tvim test path/to/file_test.vim to run tests in that file, tvim test path/to/dir to run all test files in a directory, or tvim test path/to/dir/... to run al test files in a directory and all subdirectories, excluding testdata directories.

Tests are stored as *_test.vim files, all functions matching the Test_\k+() abort signature will be run. It is customary – but not mandatory – to store n_test.vim files next to the n.vim file in the same directory.

testing.vim exposes several variables:

g:test_verbose    -v flag from commandline (0 or 1).
g:test_debug      -d flag from commandline (0 or 1).
g:test_run        -r flag from commandline, as a string.
g:test_bench      -b flag from commandline, as a string.
g:test_dir        Directory of the test file that's being run.
g:test_tmpdir     Empty temp directory for writing; this is also set as the
                  working directory before running tests.
g:test_packdir    Directory of the plugin in the temporary directory (this
                  is a link to the REAL plugin, so don't modify files!)
g:bench_n         Number of times to run target code during benchmark.

And a few functions:

Error(msg)        Add a message to v:errors.
Errorf(msg, ...)  Like Error(), but with printf() support.
Log(msg)          Add a "log message" in v:errors; this won't fail the test.
                  Useful since :echom and v:errors output isn't interleaved.
Logf(msg, ...)    Like Log, with with printf() support.
TestSyntax()      Test syntax highlighting. Usually you want to generate
                  this with test-syntax.

A test is considered to be "failed" when v:errors has any items that are not marked as informational (as done by Log()). Vim's assert_* functions write to v:errors, and it can be written to as any list. You don't need to use Error().

You can filter test functions to run with the -r option. See tvim help test for various other options.

Another very useful option is -d, which will leave the Vim instance used to run the tests open so you can use this to inspect the state. Very useful if you don't understand why a test fails.

testing.vim will run the binary form the TEST_VIM environment variable, defaulting to vim. Managing different Vim installations is out-of-scope for this project.

See e.g. jumpy.vim for an example Travis integration

Syntax highlighting tests

You can generate tests for syntax highlighting with tvim gen-syn. How it works:

  1. Write a test file for the syntax to test; for example basic.go with the contents:

    package main
    import "fmt"
    func main() {
        var msg = "Let's go!"
  2. Verify that the highlighting is correct in this file.

  3. Run tvim gen-syn syntax/test/basic.go > syntax/test/go_test.vim; this will generate a test script recording the current syntax highlighting.

  4. Test the file as any other _test.vim script.


Linting works like testing: tvim lint ./....

Currently supported lint tools:


Benchmarks are also loaded from *_test.vim files. Benchmark functions match the pattern Benchmark_\k+() abort.

Use tvim test -b to select which benchmarks to run. Use -b . to run them all.

A benchmark is expected to run the benchmarked code g:bench_n number of times.


fun! Benchmark_trim() abort
  let l:s = '  hello  '
  for i in range(0, g:bench_n)
    call gopher#internal#trim(l:s)

Syntax highlighting benchmarks

You can benchmark syntax highlighting with tvim bench-syntax file.go:666. The 666 refers to the line number to jump to before running the benchmark.

The default is to redraw! 100 times; this can be changed with an optional second argument: tvim bench-syntax file.go:666 5000.

Tips for improving performance:

  • Try to reduce the COUNT; a lot of syntax can only appear in certain places, so there is no need for Vim to look on every line. For example:

    syn region goBuildTag start="//\s*+build\s"rs=s+2 end="$"

    This works perfectly well; but build tags can only appear in comments, so this:

    syn region goBuildTag contained [..]
    syn region goComment start="//" end="$" contains=goBuildTag

    Will be faster, as Vim won't have to look for the pattern on most lines of the file, just files that are already marked as goComment.

  • Make the regexp "fail" as soon as possible, for example these two are identical in behaviour:

    syn match       goSingleDecl      /\(import\|var\|const\) [^(]\@=/
    syn match       goSingleDecl      /^\s*\(import\|var\|const\) [^(]\@=/

    But the second version is a lot faster due to the ^\s*; The regular expression can stop matching much faster for most lines.


My philosophy of testing is that it should be kept as simple as feasible; programming is already hard enough without struggling with a poorly documented unintuitive testing framework. This is what makes testing.vim different from some other Vim testing tools.

My requirements:

  1. Easy to run test, including just a single test function.
  2. Easy to see what failed exactly.
  3. Easy to debug failing tests, e.g. with printf-debugging.
  4. Reasonably intuitive for anyone familiar with VimScript.

None of the existing tools met these. Some of them didn't even meet any of them!


I originally implemented this for vim-go, based on Fatih Arslan's previous work (see 1157, 1158, and 1476), which was presumably inspired by runtest.vim in the Vim source code repository.


Comprehensive testing tool for Vim








No releases published

Sponsor this project