Permalink
Browse files

Extract testing framework from test suite

  • Loading branch information...
1 parent d250b06 commit ac6fc9367d686b8536c0f261fdc08477431c1226 @xolox committed Jun 2, 2013
Showing with 614 additions and 252 deletions.
  1. +146 −18 README.md
  2. +1 −1 autoload/xolox/misc.vim
  3. +125 −0 autoload/xolox/misc/test.vim
  4. +150 −206 autoload/xolox/misc/tests.vim
  5. +192 −27 doc/misc.txt
View
164 README.md
@@ -16,14 +16,13 @@ the miscellaneous scripts, but I don't see any way around this. Sorry!
## Installation
-Unzip the most recent [ZIP archive] [download] file inside your Vim profile
+Unzip the most recent [ZIP archive] [] file inside your Vim profile
directory (usually this is `~/.vim` on UNIX and `%USERPROFILE%\vimfiles` on
Windows), restart Vim and execute the command `:helptags ~/.vim/doc` (use
`:helptags ~\vimfiles\doc` instead on Windows).
-If you prefer you can also use [Pathogen] [pathogen], [Vundle] [vundle] or a
-similar tool to install & update the plug-in using a local clone of the git
-repository.
+If you prefer you can also use [Pathogen] [], [Vundle] [] or a similar tool to
+install & update the plug-in using a local clone of the git repository.
## Function documentation
@@ -33,13 +32,13 @@ plug-ins. I care about backwards compatibility so won't break it without a good
reason to do so.
For those who are curious: The function descriptions given below were extracted
-from the source code of the miscellaneous scripts using a bit of Python code
-that I haven't published yet.
+from the source code of the miscellaneous scripts using the Python module
+`vimdoctool.py` included in [vim-tools] [].
<!-- Start of generated documentation -->
-The documentation of the 44 functions below was extracted from
-14 Vim scripts on June 2, 2013 at 21:27.
+The documentation of the 66 functions below was extracted from
+15 Vim scripts on June 2, 2013 at 22:18.
### Handling of special buffers
@@ -421,11 +420,139 @@ Compact whitespace in the string given as the first argument.
Trim all whitespace from the start and end of the string given as the
first argument.
-### Tests for my miscellaneous Vim scripts
+### Test runner & infrastructure for Vim plug-ins
-#### The `xolox#misc#tests#run_all()` function
+The Vim auto-load script `autoload/xolox/misc/test.vim` contains
+infrastructure that can be used to run an automated Vim plug-in test suite.
+It provides a framework for running test functions, keeping track of the
+test status, making assertions and reporting test results to the user.
-Run the automated tests of the miscellaneous functions.
+#### The `xolox#misc#test#reset()` function
+
+Reset counters for executed tests and passed/failed assertions.
+
+#### The `xolox#misc#test#summarize()` function
+
+Print a summary of test results, to be interpreted interactively.
+
+#### The `xolox#misc#test#wrap()` function
+
+Call a function in a try/catch block and prevent exceptions from bubbling.
+The name of the function should be passed as the first and only argument;
+it should be a string containing the name of a Vim auto-load function.
+
+#### The `xolox#misc#test#passed()` function
+
+Record a test which succeeded.
+
+#### The `xolox#misc#test#failed()` function
+
+Record a test which failed.
+
+#### The `xolox#misc#test#assert_true()` function
+
+Check whether an expression is true.
+
+#### The `xolox#misc#test#assert_equals()` function
+
+Check whether two values are the same.
+
+#### The `xolox#misc#test#assert_same_type()` function
+
+Check whether two values are of the same type.
+
+### Tests for the miscellaneous Vim scripts
+
+The Vim auto-load script `autoload/xolox/misc/tests.vim` contains the
+automated test suite of the miscellaneous Vim scripts. Right now the
+coverage is not very high yet, but this will improve over time.
+
+#### The `xolox#misc#tests#run()` function
+
+Run the automated test suite of the miscellaneous Vim scripts. To be used
+interactively. Intended to be safe to execute irrespective of context.
+
+#### The `xolox#misc#tests#pattern_escaping()` function
+
+Test escaping of regular expression patterns with
+`xolox#misc#escape#pattern()`.
+
+#### The `xolox#misc#tests#substitute_escaping()` function
+
+Test escaping of substitution strings with
+`xolox#misc#escape#substitute()`.
+
+#### The `xolox#misc#tests#shell_escaping()` function
+
+Test escaping of shell arguments with `xolox#misc#escape#shell()`.
+
+#### The `xolox#misc#tests#making_a_list_unique()` function
+
+Test removing of duplicate values from lists with
+`xolox#misc#list#unique()`.
+
+#### The `xolox#misc#tests#binary_insertion()` function
+
+Test the binary insertion algorithm implemented in
+`xolox#misc#list#binsert()`.
+
+#### The `xolox#misc#tests#getting_configuration_options()` function
+
+Test getting of scoped plug-in configuration "options" with
+`xolox#misc#option#get()`.
+
+#### The `xolox#misc#tests#splitting_of_multi_valued_options()` function
+
+Test splitting of multi-valued Vim options with
+`xolox#misc#option#split()`.
+
+#### The `xolox#misc#tests#joining_of_multi_valued_options()` function
+
+Test joining of multi-valued Vim options with `xolox#misc#option#join()`.
+
+#### The `xolox#misc#tests#evaluation_of_tags_option()` function
+
+Test evaluation of Vim's ['tags'] [] option. We don't test `~/.tags` style
+patterns because `xolox#misc#option#eval_tags()` doesn't support those.
+Depending on your perspective this is not a bug, because the ['tags'] []
+option gets special treatment in Vim anyway:
+
+ :set tags=~/.tags
+ tags=~/.tags
+ :echo &tags
+ /home/peter/.tags
+
+So at the point where `xolox#misc#option#eval_tags()` receives the value
+of ['tags'] [], it has already been expanded by Vim.
+
+['tags']: http://vimdoc.sourceforge.net/htmldoc/options.html#'tags'
+
+#### The `xolox#misc#tests#finding_vim_on_the_search_path()` function
+
+Test looking up Vim's executable on the search path using [v:progname] []
+with `xolox#misc#os#find_vim()`.
+
+[v:progname]: http://vimdoc.sourceforge.net/htmldoc/eval.html#v:progname
+
+#### The `xolox#misc#tests#synchronous_command_execution()` function
+
+Test basic functionality of synchronous command execution with
+`xolox#misc#os#exec()`.
+
+#### The `xolox#misc#tests#synchronous_command_execution_with_raising_of_errors()` function
+
+Test raising of errors during synchronous command execution with
+`xolox#misc#os#exec()`.
+
+#### The `xolox#misc#tests#synchronous_command_execution_without_raising_errors()` function
+
+Test synchronous command execution without raising of errors with
+`xolox#misc#os#exec()`.
+
+#### The `xolox#misc#tests#asynchronous_command_execution()` function
+
+Test basic functionality of asynchronous command execution with
+`xolox#misc#os#exec()`.
### Timing of long during operations
@@ -462,18 +589,19 @@ you pass the list returned by `xolox#misc#timer#start()` as an argument.
If you have questions, bug reports, suggestions, etc. the author can be
contacted at <peter@peterodding.com>. The latest version is available at
<http://peterodding.com/code/vim/misc> and <http://github.com/xolox/vim-misc>.
-If you like the script please vote for it on [Vim Online] [vim-online].
+If you like the script please vote for it on [Vim Online] [].
## License
-This software is licensed under the [MIT license] [mit].
+This software is licensed under the [MIT license] [].
© 2013 Peter Odding &lt;<peter@peterodding.com>&gt;.
-[download]: http://peterodding.com/code/vim/downloads/misc.zip
-[mit]: http://en.wikipedia.org/wiki/MIT_License
-[pathogen]: http://www.vim.org/scripts/script.php?script_id=2332
+[MIT license]: http://en.wikipedia.org/wiki/MIT_License
+[Pathogen]: http://www.vim.org/scripts/script.php?script_id=2332
[plugins]: http://peterodding.com/code/vim/
[repository]: https://github.com/xolox/vim-misc
-[vim-online]: http://www.vim.org/scripts/script.php?script_id=4597
-[vundle]: https://github.com/gmarik/vundle
+[Vim Online]: http://www.vim.org/scripts/script.php?script_id=4597
+[vim-tools]: http://peterodding.com/code/vim/tools/
+[Vundle]: https://github.com/gmarik/vundle
+[ZIP archive]: http://peterodding.com/code/vim/downloads/misc.zip
View
@@ -4,4 +4,4 @@
" Last Change: June 2, 2013
" URL: http://peterodding.com/code/vim/misc/
-let g:xolox#misc#version = '1.3'
+let g:xolox#misc#version = '1.4'
@@ -0,0 +1,125 @@
+" Test runner & infrastructure for Vim plug-ins.
+"
+" Author: Peter Odding <peter@peterodding.com>
+" Last Change: June 2, 2013
+" URL: http://peterodding.com/code/vim/misc/
+"
+" The Vim auto-load script `autoload/xolox/misc/test.vim` contains
+" infrastructure that can be used to run an automated Vim plug-in test suite.
+" It provides a framework for running test functions, keeping track of the
+" test status, making assertions and reporting test results to the user.
+
+" The process handling tests cannot use the built-in "echo" command from the
+" Windows shell because it has way too much idiosyncrasies for me to put up
+" with. Seriously. Instead I'm using an "echo.exe" from the UnxUtils project.
+if xolox#misc#os#is_win()
+ let g:xolox#misc#test#echo = xolox#misc#escape#shell(xolox#misc#path#merge(expand('<sfile>:p:h'), 'echo.exe'))
+else
+ let g:xolox#misc#test#echo = 'echo'
+endif
+
+function! xolox#misc#test#reset() " {{{1
+ " Reset counters for executed tests and passed/failed assertions.
+ let s:num_executed = 0
+ let s:num_passed = 0
+ let s:num_failed = 0
+ let s:tests_started_at = xolox#misc#timer#start()
+endfunction
+
+function! xolox#misc#test#summarize() " {{{1
+ " Print a summary of test results, to be interpreted interactively.
+ call s:delimit_output()
+ call xolox#misc#timer#force("Took %s to run %s: %s passed, %s failed.",
+ \ s:tests_started_at,
+ \ xolox#misc#format#pluralize(s:num_executed, 'test', 'tests'),
+ \ xolox#misc#format#pluralize(s:num_passed, 'assertion', 'assertions'),
+ \ xolox#misc#format#pluralize(s:num_failed, 'assertion', 'assertions'))
+endfunction
+
+function! xolox#misc#test#wrap(function) " {{{1
+ " Call a function in a try/catch block and prevent exceptions from bubbling.
+ " The name of the function should be passed as the first and only argument;
+ " it should be a string containing the name of a Vim auto-load function.
+ let num_failed = s:num_failed
+ try
+ if s:num_passed + s:num_failed > 0
+ call s:delimit_output()
+ endif
+ let test_name = split(a:function, '#')[-1]
+ let test_name = substitute(test_name, '_', ' ', 'g')
+ let test_name = substitute(test_name, '^.', '\U\0', '')
+ call xolox#misc#msg#info("Running test #%i: %s", s:num_executed + 1, test_name)
+ call call(a:function, [])
+ catch
+ call xolox#misc#msg#warn("Test %s raised exception:", a:function)
+ call xolox#misc#msg#warn("%s", v:exception)
+ call xolox#misc#msg#warn("(at %s)", v:throwpoint)
+ if num_failed == s:num_failed
+ " Make sure exceptions are counted as failures, but don't inflate the
+ " number of failed assertions when it's not needed (it can produce
+ " confusing test output).
+ call xolox#misc#test#failed()
+ endif
+ endtry
+ let s:num_executed += 1
+endfunction
+
+function! xolox#misc#test#passed() " {{{1
+ " Record a test which succeeded.
+ let s:num_passed += 1
+ call s:print_feedback()
+endfunction
+
+function! xolox#misc#test#failed() " {{{1
+ " Record a test which failed.
+ let s:num_failed += 1
+ call s:print_feedback()
+endfunction
+
+function! s:delimit_output() " {{{1
+ " Print a delimiter between output of tests.
+ call xolox#misc#msg#info("%s", repeat("-", 40))
+endfunction
+
+function! s:print_feedback() " {{{1
+ " Let the user know the status of the test suite.
+ call xolox#misc#msg#info("Test status: %s passed, %s failed ..",
+ \ xolox#misc#format#pluralize(s:num_passed, 'assertion', 'assertions'),
+ \ xolox#misc#format#pluralize(s:num_failed, 'assertion', 'assertions'))
+endfunction
+
+function! xolox#misc#test#assert_true(expr) " {{{1
+ " Check whether an expression is true.
+ if a:expr
+ call xolox#misc#test#passed()
+ else
+ call xolox#misc#test#failed()
+ let msg = "Expected value to be true, got %s instead"
+ throw printf(msg, string(a:expr))
+ endif
+endfunction
+
+function! xolox#misc#test#assert_equals(expected, received) " {{{1
+ " Check whether two values are the same.
+ call xolox#misc#test#assert_same_type(a:expected, a:received)
+ if a:expected == a:received
+ call xolox#misc#test#passed()
+ else
+ call xolox#misc#test#failed()
+ let msg = "Expected value %s, received value %s!"
+ throw printf(msg, string(a:expected), string(a:received))
+ endif
+endfunction
+
+function! xolox#misc#test#assert_same_type(expected, received) " {{{1
+ " Check whether two values are of the same type.
+ if type(a:expected) == type(a:received)
+ call xolox#misc#test#passed()
+ else
+ call xolox#misc#test#failed()
+ let msg = "Expected value of same type as %s, got value %s!"
+ throw printf(msg, string(a:expected), string(a:received))
+ endif
+endfunction
+
+call xolox#misc#test#reset()
Oops, something went wrong.

0 comments on commit ac6fc93

Please sign in to comment.