Skip to content

Commit

Permalink
Extract testing framework from test suite
Browse files Browse the repository at this point in the history
  • Loading branch information
xolox committed Jun 2, 2013
1 parent d250b06 commit ac6fc93
Show file tree
Hide file tree
Showing 5 changed files with 614 additions and 252 deletions.
164 changes: 146 additions & 18 deletions README.md
Expand Up @@ -16,14 +16,13 @@ the miscellaneous scripts, but I don't see any way around this. Sorry!

## Installation

Unzip the most recent [ZIP archive] [download] file inside your Vim profile
Unzip the most recent [ZIP archive] [] file inside your Vim profile
directory (usually this is `~/.vim` on UNIX and `%USERPROFILE%\vimfiles` on
Windows), restart Vim and execute the command `:helptags ~/.vim/doc` (use
`:helptags ~\vimfiles\doc` instead on Windows).

If you prefer you can also use [Pathogen] [pathogen], [Vundle] [vundle] or a
similar tool to install & update the plug-in using a local clone of the git
repository.
If you prefer you can also use [Pathogen] [], [Vundle] [] or a similar tool to
install & update the plug-in using a local clone of the git repository.

## Function documentation

Expand All @@ -33,13 +32,13 @@ plug-ins. I care about backwards compatibility so won't break it without a good
reason to do so.

For those who are curious: The function descriptions given below were extracted
from the source code of the miscellaneous scripts using a bit of Python code
that I haven't published yet.
from the source code of the miscellaneous scripts using the Python module
`vimdoctool.py` included in [vim-tools] [].

<!-- Start of generated documentation -->

The documentation of the 44 functions below was extracted from
14 Vim scripts on June 2, 2013 at 21:27.
The documentation of the 66 functions below was extracted from
15 Vim scripts on June 2, 2013 at 22:18.

### Handling of special buffers

Expand Down Expand Up @@ -421,11 +420,139 @@ Compact whitespace in the string given as the first argument.
Trim all whitespace from the start and end of the string given as the
first argument.

### Tests for my miscellaneous Vim scripts
### Test runner & infrastructure for Vim plug-ins

#### The `xolox#misc#tests#run_all()` function
The Vim auto-load script `autoload/xolox/misc/test.vim` contains
infrastructure that can be used to run an automated Vim plug-in test suite.
It provides a framework for running test functions, keeping track of the
test status, making assertions and reporting test results to the user.

Run the automated tests of the miscellaneous functions.
#### The `xolox#misc#test#reset()` function

Reset counters for executed tests and passed/failed assertions.

#### The `xolox#misc#test#summarize()` function

Print a summary of test results, to be interpreted interactively.

#### The `xolox#misc#test#wrap()` function

Call a function in a try/catch block and prevent exceptions from bubbling.
The name of the function should be passed as the first and only argument;
it should be a string containing the name of a Vim auto-load function.

#### The `xolox#misc#test#passed()` function

Record a test which succeeded.

#### The `xolox#misc#test#failed()` function

Record a test which failed.

#### The `xolox#misc#test#assert_true()` function

Check whether an expression is true.

#### The `xolox#misc#test#assert_equals()` function

Check whether two values are the same.

#### The `xolox#misc#test#assert_same_type()` function

Check whether two values are of the same type.

### Tests for the miscellaneous Vim scripts

The Vim auto-load script `autoload/xolox/misc/tests.vim` contains the
automated test suite of the miscellaneous Vim scripts. Right now the
coverage is not very high yet, but this will improve over time.

#### The `xolox#misc#tests#run()` function

Run the automated test suite of the miscellaneous Vim scripts. To be used
interactively. Intended to be safe to execute irrespective of context.

#### The `xolox#misc#tests#pattern_escaping()` function

Test escaping of regular expression patterns with
`xolox#misc#escape#pattern()`.

#### The `xolox#misc#tests#substitute_escaping()` function

Test escaping of substitution strings with
`xolox#misc#escape#substitute()`.

#### The `xolox#misc#tests#shell_escaping()` function

Test escaping of shell arguments with `xolox#misc#escape#shell()`.

#### The `xolox#misc#tests#making_a_list_unique()` function

Test removing of duplicate values from lists with
`xolox#misc#list#unique()`.

#### The `xolox#misc#tests#binary_insertion()` function

Test the binary insertion algorithm implemented in
`xolox#misc#list#binsert()`.

#### The `xolox#misc#tests#getting_configuration_options()` function

Test getting of scoped plug-in configuration "options" with
`xolox#misc#option#get()`.

#### The `xolox#misc#tests#splitting_of_multi_valued_options()` function

Test splitting of multi-valued Vim options with
`xolox#misc#option#split()`.

#### The `xolox#misc#tests#joining_of_multi_valued_options()` function

Test joining of multi-valued Vim options with `xolox#misc#option#join()`.

#### The `xolox#misc#tests#evaluation_of_tags_option()` function

Test evaluation of Vim's ['tags'] [] option. We don't test `~/.tags` style
patterns because `xolox#misc#option#eval_tags()` doesn't support those.
Depending on your perspective this is not a bug, because the ['tags'] []
option gets special treatment in Vim anyway:

:set tags=~/.tags
tags=~/.tags
:echo &tags
/home/peter/.tags

So at the point where `xolox#misc#option#eval_tags()` receives the value
of ['tags'] [], it has already been expanded by Vim.

['tags']: http://vimdoc.sourceforge.net/htmldoc/options.html#'tags'

#### The `xolox#misc#tests#finding_vim_on_the_search_path()` function

Test looking up Vim's executable on the search path using [v:progname] []
with `xolox#misc#os#find_vim()`.

[v:progname]: http://vimdoc.sourceforge.net/htmldoc/eval.html#v:progname

#### The `xolox#misc#tests#synchronous_command_execution()` function

Test basic functionality of synchronous command execution with
`xolox#misc#os#exec()`.

#### The `xolox#misc#tests#synchronous_command_execution_with_raising_of_errors()` function

Test raising of errors during synchronous command execution with
`xolox#misc#os#exec()`.

#### The `xolox#misc#tests#synchronous_command_execution_without_raising_errors()` function

Test synchronous command execution without raising of errors with
`xolox#misc#os#exec()`.

#### The `xolox#misc#tests#asynchronous_command_execution()` function

Test basic functionality of asynchronous command execution with
`xolox#misc#os#exec()`.

### Timing of long during operations

Expand Down Expand Up @@ -462,18 +589,19 @@ you pass the list returned by `xolox#misc#timer#start()` as an argument.
If you have questions, bug reports, suggestions, etc. the author can be
contacted at <peter@peterodding.com>. The latest version is available at
<http://peterodding.com/code/vim/misc> and <http://github.com/xolox/vim-misc>.
If you like the script please vote for it on [Vim Online] [vim-online].
If you like the script please vote for it on [Vim Online] [].

## License

This software is licensed under the [MIT license] [mit].
This software is licensed under the [MIT license] [].
© 2013 Peter Odding &lt;<peter@peterodding.com>&gt;.


[download]: http://peterodding.com/code/vim/downloads/misc.zip
[mit]: http://en.wikipedia.org/wiki/MIT_License
[pathogen]: http://www.vim.org/scripts/script.php?script_id=2332
[MIT license]: http://en.wikipedia.org/wiki/MIT_License
[Pathogen]: http://www.vim.org/scripts/script.php?script_id=2332
[plugins]: http://peterodding.com/code/vim/
[repository]: https://github.com/xolox/vim-misc
[vim-online]: http://www.vim.org/scripts/script.php?script_id=4597
[vundle]: https://github.com/gmarik/vundle
[Vim Online]: http://www.vim.org/scripts/script.php?script_id=4597
[vim-tools]: http://peterodding.com/code/vim/tools/
[Vundle]: https://github.com/gmarik/vundle
[ZIP archive]: http://peterodding.com/code/vim/downloads/misc.zip
2 changes: 1 addition & 1 deletion autoload/xolox/misc.vim
Expand Up @@ -4,4 +4,4 @@
" Last Change: June 2, 2013
" URL: http://peterodding.com/code/vim/misc/

let g:xolox#misc#version = '1.3'
let g:xolox#misc#version = '1.4'
125 changes: 125 additions & 0 deletions autoload/xolox/misc/test.vim
@@ -0,0 +1,125 @@
" Test runner & infrastructure for Vim plug-ins.
"
" Author: Peter Odding <peter@peterodding.com>
" Last Change: June 2, 2013
" URL: http://peterodding.com/code/vim/misc/
"
" The Vim auto-load script `autoload/xolox/misc/test.vim` contains
" infrastructure that can be used to run an automated Vim plug-in test suite.
" It provides a framework for running test functions, keeping track of the
" test status, making assertions and reporting test results to the user.

" The process handling tests cannot use the built-in "echo" command from the
" Windows shell because it has way too much idiosyncrasies for me to put up
" with. Seriously. Instead I'm using an "echo.exe" from the UnxUtils project.
if xolox#misc#os#is_win()
let g:xolox#misc#test#echo = xolox#misc#escape#shell(xolox#misc#path#merge(expand('<sfile>:p:h'), 'echo.exe'))
else
let g:xolox#misc#test#echo = 'echo'
endif

function! xolox#misc#test#reset() " {{{1
" Reset counters for executed tests and passed/failed assertions.
let s:num_executed = 0
let s:num_passed = 0
let s:num_failed = 0
let s:tests_started_at = xolox#misc#timer#start()
endfunction

function! xolox#misc#test#summarize() " {{{1
" Print a summary of test results, to be interpreted interactively.
call s:delimit_output()
call xolox#misc#timer#force("Took %s to run %s: %s passed, %s failed.",
\ s:tests_started_at,
\ xolox#misc#format#pluralize(s:num_executed, 'test', 'tests'),
\ xolox#misc#format#pluralize(s:num_passed, 'assertion', 'assertions'),
\ xolox#misc#format#pluralize(s:num_failed, 'assertion', 'assertions'))
endfunction

function! xolox#misc#test#wrap(function) " {{{1
" Call a function in a try/catch block and prevent exceptions from bubbling.
" The name of the function should be passed as the first and only argument;
" it should be a string containing the name of a Vim auto-load function.
let num_failed = s:num_failed
try
if s:num_passed + s:num_failed > 0
call s:delimit_output()
endif
let test_name = split(a:function, '#')[-1]
let test_name = substitute(test_name, '_', ' ', 'g')
let test_name = substitute(test_name, '^.', '\U\0', '')
call xolox#misc#msg#info("Running test #%i: %s", s:num_executed + 1, test_name)
call call(a:function, [])
catch
call xolox#misc#msg#warn("Test %s raised exception:", a:function)
call xolox#misc#msg#warn("%s", v:exception)
call xolox#misc#msg#warn("(at %s)", v:throwpoint)
if num_failed == s:num_failed
" Make sure exceptions are counted as failures, but don't inflate the
" number of failed assertions when it's not needed (it can produce
" confusing test output).
call xolox#misc#test#failed()
endif
endtry
let s:num_executed += 1
endfunction

function! xolox#misc#test#passed() " {{{1
" Record a test which succeeded.
let s:num_passed += 1
call s:print_feedback()
endfunction

function! xolox#misc#test#failed() " {{{1
" Record a test which failed.
let s:num_failed += 1
call s:print_feedback()
endfunction

function! s:delimit_output() " {{{1
" Print a delimiter between output of tests.
call xolox#misc#msg#info("%s", repeat("-", 40))
endfunction

function! s:print_feedback() " {{{1
" Let the user know the status of the test suite.
call xolox#misc#msg#info("Test status: %s passed, %s failed ..",
\ xolox#misc#format#pluralize(s:num_passed, 'assertion', 'assertions'),
\ xolox#misc#format#pluralize(s:num_failed, 'assertion', 'assertions'))
endfunction

function! xolox#misc#test#assert_true(expr) " {{{1
" Check whether an expression is true.
if a:expr
call xolox#misc#test#passed()
else
call xolox#misc#test#failed()
let msg = "Expected value to be true, got %s instead"
throw printf(msg, string(a:expr))
endif
endfunction

function! xolox#misc#test#assert_equals(expected, received) " {{{1
" Check whether two values are the same.
call xolox#misc#test#assert_same_type(a:expected, a:received)
if a:expected == a:received
call xolox#misc#test#passed()
else
call xolox#misc#test#failed()
let msg = "Expected value %s, received value %s!"
throw printf(msg, string(a:expected), string(a:received))
endif
endfunction

function! xolox#misc#test#assert_same_type(expected, received) " {{{1
" Check whether two values are of the same type.
if type(a:expected) == type(a:received)
call xolox#misc#test#passed()
else
call xolox#misc#test#failed()
let msg = "Expected value of same type as %s, got value %s!"
throw printf(msg, string(a:expected), string(a:received))
endif
endfunction

call xolox#misc#test#reset()

0 comments on commit ac6fc93

Please sign in to comment.