Skip to content

Commit

Permalink
Make external command execution work properly without vim-shell!
Browse files Browse the repository at this point in the history 
In vim-easytags/issues#58 it became apparent that external command
execution on Windows using vim-misc without vim-shell was completely
broken :-(

I never noticed before because I always have vim-shell installed and
honestly just never gave the interaction any thought... Crap.

This commit fixes xolox#misc#os#exec() on Windows when vim-shell is
not available. It also adds testing infrastructure to run all tests
involving xolox#misc#os#exec() twice, once with vim-shell disabled
and then again with vim-shell enabled (assuming it is available).

For more details, see the issue page on GitHub:
  xolox/vim-easytags#58
  • Loading branch information
@xolox
xolox committed Jun 25, 2013
1 parent 473381c commit d780f0a
Show file tree
Hide file tree
Showing 5 changed files with 191 additions and 73 deletions.
24 changes: 20 additions & 4 deletions README.md
View file
Expand Up @@ -37,8 +37,8 @@ from the source code of the miscellaneous scripts using the Python module

<!-- Start of generated documentation -->

The documentation of the 78 functions below was extracted from
15 Vim scripts on June 25, 2013 at 23:45.
The documentation of the 80 functions below was extracted from
15 Vim scripts on June 26, 2013 at 00:09.

### Handling of special buffers

Expand Down Expand Up @@ -337,6 +337,13 @@ Returns a dictionary with one or more of the following key/value pairs:

[vim-shell]: http://peterodding.com/code/vim/shell/

#### The `xolox#misc#os#can_use_dll()` function

If a) we're on Microsoft Windows, b) the vim-shell plug-in is installed
and c) the compiled DLL included in vim-shell works, we can use the
vim-shell plug-in to execute external commands! Returns 1 (true)
if we can use the DLL, 0 (false) otherwise.

### Pathname manipulation functions

#### The `xolox#misc#path#which()` function
Expand Down Expand Up @@ -553,6 +560,12 @@ with `xolox#misc#os#find_vim()`.
Test basic functionality of synchronous command execution with
`xolox#misc#os#exec()`.

#### The `xolox#misc#tests#synchronous_command_execution_with_stderr()` function

Test basic functionality of synchronous command execution with
`xolox#misc#os#exec()` including the standard error stream (not available
on Windows when vim-shell is not installed).

#### The `xolox#misc#tests#synchronous_command_execution_with_raising_of_errors()` function

Test raising of errors during synchronous command execution with
Expand All @@ -565,8 +578,11 @@ Test synchronous command execution without raising of errors with

#### The `xolox#misc#tests#asynchronous_command_execution()` function

Test basic functionality of asynchronous command execution with
`xolox#misc#os#exec()`.
Test the basic functionality of asynchronous command execution with
`xolox#misc#os#exec()`. This runs the external command `mkdir` and tests
that the side effect of creating the directory takes place. This might
seem like a peculiar choice, but it's one of the few 100% portable
commands (Windows + UNIX) that doesn't involve input/output streams.

#### The `xolox#misc#tests#string_case_transformation()` function

Expand Down
2 changes: 1 addition & 1 deletion autoload/xolox/misc.vim
View file
Expand Up @@ -4,4 +4,4 @@
" Last Change: June 25, 2013
" URL: http://peterodding.com/code/vim/misc/

let g:xolox#misc#version = '1.8.2'
let g:xolox#misc#version = '1.8.3'
112 changes: 77 additions & 35 deletions autoload/xolox/misc/os.vim
View file
Expand Up @@ -113,8 +113,24 @@ function! xolox#misc#os#exec(options) " {{{1
let cmd = a:options['command']
let async = get(a:options, 'async', 0)

" We need to know in a couple of places whether we are on Windows.
let is_win = xolox#misc#os#is_win()

" Use vim-shell so we don't pop up a console window on Windows? If the
" caller specifically asks us *not* to use vim-shell, we'll respect that
" choice; this is very useful for automated tests :-).
if get(a:options, 'use_dll', 1) == 0
let use_dll = 0
else
let use_dll = xolox#misc#os#can_use_dll()
endif

" Decide whether to redirect the standard output and standard error
" streams to temporary files.
let redirect_output = !async && (use_dll || !is_win)

" Write the input for the external command to a temporary file?
if has_key(a:options, 'stdin')
if has_key(a:options, 'stdin') && use_dll
let tempin = tempname()
if type(a:options['stdin']) == type([])
let lines = a:options['stdin']
Expand All @@ -125,68 +141,82 @@ function! xolox#misc#os#exec(options) " {{{1
let cmd .= ' < ' . xolox#misc#escape#shell(tempin)
endif

" Redirect the standard output and standard error streams of the external
" process to temporary files? (only in synchronous mode, which is the
" default).
if !async
" Redirect the standard output and/or standard error streams of the
" external process to temporary files? (only in synchronous mode)
if redirect_output
let tempout = tempname()
let temperr = tempname()
let cmd = printf('(%s) 1>%s 2>%s', cmd, xolox#misc#escape#shell(tempout), xolox#misc#escape#shell(temperr))
endif

" If A) we're on Windows, B) the vim-shell plug-in is installed and C) the
" compiled DLL works, we'll use that because it's the most user friendly
" method. If the plug-in is not installed Vim will raise the exception
" "E117: Unknown function" which is caught and handled below.
try
if xolox#shell#can_use_dll()
" Let the user know what's happening (in case they're interested).
call xolox#misc#msg#debug("vim-misc %s: Executing external command using compiled DLL: %s", g:xolox#misc#version, cmd)
let exit_code = xolox#shell#execute_with_dll(cmd, async)
endif
catch /^Vim\%((\a\+)\)\=:E117/
call xolox#misc#msg#debug("vim-misc %s: The vim-shell plug-in is not installed, falling back to system() function.", g:xolox#misc#version)
endtry

" If we cannot use the DLL, we fall back to the default and generic
" implementation using Vim's system() function.
if !exists('exit_code')
" Use vim-shell or system() to execute the external command?
if use_dll
call xolox#misc#msg#debug("vim-misc %s: Executing external command using compiled DLL: %s", g:xolox#misc#version, cmd)
let exit_code = xolox#shell#execute_with_dll(cmd, async)
else

" Enable asynchronous mode (very platform specific).
if async
if xolox#misc#os#is_win()
let cmd = 'start /b ' . cmd
if is_win
let cmd = printf('start /b %s', cmd)
elseif has('unix')
let cmd = '(' . cmd . ') &'
let cmd = printf('(%s) &', cmd)
else
call xolox#misc#msg#warn("vim-misc %s: I don't know how to run commands asynchronously on your platform! Falling back to synchronous mode.", g:xolox#misc#version)
call xolox#misc#msg#warn("vim-misc %s: I don't know how to execute the command %s asynchronously on your platform! Falling back to synchronous mode...", g:xolox#misc#version, cmd)
endif
endif

" Execute the command line using 'sh' instead of the default shell,
" because we assume that standard output and standard error can be
" redirected separately, but (t)csh does not support this.
" On UNIX we explicitly execute the command line using 'sh' instead of
" the default shell, because we assume that standard output and standard
" error can be redirected separately, but (t)csh does not support this
" (and it might be the default shell).
if has('unix')
call xolox#misc#msg#debug("vim-misc %s: Generated shell expression: %s", g:xolox#misc#version, cmd)
let cmd = printf('sh -c %s', xolox#misc#escape#shell(cmd))
endif

" Let the user know what's happening (in case they're interested).
call xolox#misc#msg#debug("vim-misc %s: Executing external command using system() function: %s", g:xolox#misc#version, cmd)
call system(cmd)
let exit_code = v:shell_error
if async && is_win
call xolox#misc#msg#debug("vim-misc %s: Executing external command using !start command: %s", g:xolox#misc#version, cmd)
silent execute '!' . cmd
else
call xolox#misc#msg#debug("vim-misc %s: Executing external command using system() function: %s", g:xolox#misc#version, cmd)
let arguments = [cmd]
if has_key(a:options, 'stdin')
if type(a:options['stdin']) == type([])
call add(arguments, join(a:options['stdin'], "\n"))
else
call add(arguments, a:options['stdin'])
endif
endif
let stdout = call('system', arguments)
let exit_code = v:shell_error
endif

endif

" Return the results as a dictionary with one or more key/value pairs.
let result = {'command': cmd}
if !async
let result['exit_code'] = exit_code
let result['stdout'] = s:readfile(tempout, 'standard output', a:options['command'])
let result['stderr'] = s:readfile(temperr, 'standard error', a:options['command'])
" Get the standard output of the command.
if redirect_output
let result['stdout'] = s:readfile(tempout, 'standard output', a:options['command'])
elseif exists('stdout')
let result['stdout'] = split(stdout, "\n")
else
let result['stdout'] = []
endif
" Get the standard error of the command.
if exists('temperr')
let result['stderr'] = s:readfile(temperr, 'standard error', a:options['command'])
else
let result['stderr'] = []
endif
" If we just executed a synchronous command and the caller didn't
" specifically ask us *not* to check the exit code of the external
" command, we'll do so now.
" command, we'll do so now. The idea here is that it should be easy
" to 'do the right thing'.
if get(a:options, 'check', 1) && exit_code != 0
" Prepare an error message with enough details so the user can investigate.
let msg = printf("vim-misc %s: External command failed with exit code %d!", g:xolox#misc#version, result['exit_code'])
Expand Down Expand Up @@ -215,6 +245,18 @@ function! xolox#misc#os#exec(options) " {{{1

endfunction

function! xolox#misc#os#can_use_dll() " {{{1
" If a) we're on Microsoft Windows, b) the vim-shell plug-in is installed
" and c) the compiled DLL included in vim-shell works, we can use the
" vim-shell plug-in to execute external commands! Returns 1 (true)
" if we can use the DLL, 0 (false) otherwise.
try
return xolox#shell#can_use_dll()
catch /^Vim\%((\a\+)\)\=:E117/
return 0
endtry
endfunction

function! s:readfile(fname, label, cmd) " {{{1
try
return readfile(a:fname)
Expand Down
80 changes: 60 additions & 20 deletions autoload/xolox/misc/tests.vim
View file
Expand Up @@ -8,6 +8,9 @@
" automated test suite of the miscellaneous Vim scripts. Right now the
" coverage is not very high yet, but this will improve over time.

let s:use_dll = 0
let s:can_use_dll = xolox#misc#os#can_use_dll()

function! xolox#misc#tests#run() " {{{1
" Run the automated test suite of the miscellaneous Vim scripts. To be used
" interactively. Intended to be safe to execute irrespective of context.
Expand All @@ -23,12 +26,27 @@ function! xolox#misc#tests#run() " {{{1
call xolox#misc#test#summarize()
endfunction

function! s:wrap_exec_test(function)
" Wrapper for tests that use xolox#misc#os#exec(). If we're on Windows and
" the vim-shell plug-in is installed, the test will be run twice: Once with
" vim-shell disabled and once with vim-shell enabled. This makes sure that
" all code paths are tested as much as possible.
call xolox#misc#msg#debug("vim-misc %s: Temporarily disabling vim-shell so we can test vim-misc ..", g:xolox#misc#version)
let s:use_dll = 0
call xolox#misc#test#wrap(a:function)
if s:can_use_dll
call xolox#misc#msg#debug("vim-misc %s: Re-enabling vim-shell so we can test that as well ..", g:xolox#misc#version)
let s:use_dll = 1
call xolox#misc#test#wrap(a:function)
endif
endfunction

" Tests for autoload/xolox/misc/escape.vim {{{1

function! s:test_string_escaping()
call xolox#misc#test#wrap('xolox#misc#tests#pattern_escaping')
call xolox#misc#test#wrap('xolox#misc#tests#substitute_escaping')
call xolox#misc#test#wrap('xolox#misc#tests#shell_escaping')
call s:wrap_exec_test('xolox#misc#tests#shell_escaping')
endfunction

function! xolox#misc#tests#pattern_escaping() " {{{2
Expand All @@ -47,9 +65,15 @@ endfunction
function! xolox#misc#tests#shell_escaping() " {{{2
" Test escaping of shell arguments with `xolox#misc#escape#shell()`.
let expected_value = 'this < is > a | very " scary ^ string '' indeed'
let result = xolox#misc#os#exec({'command': g:xolox#misc#test#echo . ' ' . xolox#misc#escape#shell(expected_value)})
let result = xolox#misc#os#exec({'command': g:xolox#misc#test#echo . ' ' . xolox#misc#escape#shell(expected_value), 'use_dll': s:use_dll})
call xolox#misc#test#assert_equals(0, result['exit_code'])
call xolox#misc#test#assert_equals(0, result['exit_code'])
call xolox#misc#test#assert_equals([expected_value], result['stdout'])
call xolox#misc#test#assert_same_type([], result['stdout'])
call xolox#misc#test#assert_equals(1, len(result['stdout']))
" XXX On Windows using system() there's a trailing space I can't explain.
" However the point of this test was to show that all characters pass
" through unharmed, so for now I'll just ignore the space :-)
call xolox#misc#test#assert_equals(expected_value, xolox#misc#str#trim(result['stdout'][0]))
endfunction

" Tests for autoload/xolox/misc/list.vim {{{1
Expand Down Expand Up @@ -136,10 +160,11 @@ endfunction

function! s:test_command_execution()
call xolox#misc#test#wrap('xolox#misc#tests#finding_vim_on_the_search_path')
call xolox#misc#test#wrap('xolox#misc#tests#synchronous_command_execution')
call xolox#misc#test#wrap('xolox#misc#tests#synchronous_command_execution_with_raising_of_errors')
call xolox#misc#test#wrap('xolox#misc#tests#synchronous_command_execution_without_raising_errors')
call xolox#misc#test#wrap('xolox#misc#tests#asynchronous_command_execution')
call s:wrap_exec_test('xolox#misc#tests#synchronous_command_execution')
call s:wrap_exec_test('xolox#misc#tests#synchronous_command_execution_with_stderr')
call s:wrap_exec_test('xolox#misc#tests#synchronous_command_execution_with_raising_of_errors')
call s:wrap_exec_test('xolox#misc#tests#synchronous_command_execution_without_raising_errors')
call s:wrap_exec_test('xolox#misc#tests#asynchronous_command_execution')
endfunction

function! xolox#misc#tests#finding_vim_on_the_search_path() " {{{2
Expand All @@ -155,18 +180,30 @@ endfunction
function! xolox#misc#tests#synchronous_command_execution() " {{{2
" Test basic functionality of synchronous command execution with
" `xolox#misc#os#exec()`.
let result = xolox#misc#os#exec({'command': printf('%s output && %s errors >&2', g:xolox#misc#test#echo, g:xolox#misc#test#echo)})
let result = xolox#misc#os#exec({'command': printf('%s output', g:xolox#misc#test#echo), 'use_dll': s:use_dll})
call xolox#misc#test#assert_same_type({}, result)
call xolox#misc#test#assert_equals(0, result['exit_code'])
call xolox#misc#test#assert_equals(['output'], result['stdout'])
call xolox#misc#test#assert_equals(['errors'], result['stderr'])
endfunction

function! xolox#misc#tests#synchronous_command_execution_with_stderr() " {{{2
" Test basic functionality of synchronous command execution with
" `xolox#misc#os#exec()` including the standard error stream (not available
" on Windows when vim-shell is not installed).
if !(xolox#misc#os#is_win() && !s:use_dll)
let result = xolox#misc#os#exec({'command': printf('%s output && %s errors >&2', g:xolox#misc#test#echo, g:xolox#misc#test#echo), 'use_dll': s:use_dll})
call xolox#misc#test#assert_same_type({}, result)
call xolox#misc#test#assert_equals(0, result['exit_code'])
call xolox#misc#test#assert_equals(['output'], result['stdout'])
call xolox#misc#test#assert_equals(['errors'], result['stderr'])
endif
endfunction

function! xolox#misc#tests#synchronous_command_execution_with_raising_of_errors() " {{{2
" Test raising of errors during synchronous command execution with
" `xolox#misc#os#exec()`.
try
call xolox#misc#os#exec({'command': 'exit 1'})
call xolox#misc#os#exec({'command': 'exit 1', 'use_dll': s:use_dll})
call xolox#misc#test#assert_true(0)
catch
call xolox#misc#test#assert_true(1)
Expand All @@ -177,7 +214,7 @@ function! xolox#misc#tests#synchronous_command_execution_without_raising_errors(
" Test synchronous command execution without raising of errors with
" `xolox#misc#os#exec()`.
try
let result = xolox#misc#os#exec({'command': 'exit 42', 'check': 0})
let result = xolox#misc#os#exec({'command': 'exit 42', 'check': 0, 'use_dll': s:use_dll})
call xolox#misc#test#assert_true(1)
call xolox#misc#test#assert_equals(42, result['exit_code'])
catch
Expand All @@ -186,20 +223,23 @@ function! xolox#misc#tests#synchronous_command_execution_without_raising_errors(
endfunction

function! xolox#misc#tests#asynchronous_command_execution() " {{{2
" Test basic functionality of asynchronous command execution with
" `xolox#misc#os#exec()`.
let tempfile = tempname()
let expected_value = string(localtime())
let command = g:xolox#misc#test#echo . ' ' . xolox#misc#escape#shell(expected_value) . ' > ' . tempfile
let result = xolox#misc#os#exec({'command': command, 'async': 1})
" Test the basic functionality of asynchronous command execution with
" `xolox#misc#os#exec()`. This runs the external command `mkdir` and tests
" that the side effect of creating the directory takes place. This might
" seem like a peculiar choice, but it's one of the few 100% portable
" commands (Windows + UNIX) that doesn't involve input/output streams.
let temporary_directory = xolox#misc#path#tempdir()
let random_name = printf('%i', localtime())
let expected_directory = xolox#misc#path#merge(temporary_directory, random_name)
let command = 'mkdir ' . xolox#misc#escape#shell(expected_directory)
let result = xolox#misc#os#exec({'command': command, 'async': 1, 'use_dll': s:use_dll})
call xolox#misc#test#assert_same_type({}, result)
" Make sure the command is really executed.
let timeout = localtime() + 30
while !filereadable(tempfile) && localtime() < timeout
while !isdirectory(expected_directory) && localtime() < timeout
sleep 500 m
endwhile
call xolox#misc#test#assert_true(filereadable(tempfile))
call xolox#misc#test#assert_equals([expected_value], readfile(tempfile))
call xolox#misc#test#assert_true(isdirectory(expected_directory))
endfunction

" Tests for autoload/xolox/misc/str.vim {{{1
Expand Down

1 comment on commit d780f0a

@xolox
Copy link
Owner Author

@xolox xolox commented on d780f0a Jun 25, 2013

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I keep getting the GitHub issue references wrong... I wish @github would just recognize the absolute URLs I paste in there anyway. Now I'm wondering, maybe this comment will allow me to sneak in a proper reference after the fact...

Let's see: xolox/vim-easytags#58.

Edit: \o/ it works. Thank you @github :-)

Please sign in to comment.