Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variables set in one test affect other tests #101

Closed
HiPhish opened this issue Dec 13, 2016 · 1 comment
Closed

Variables set in one test affect other tests #101

HiPhish opened this issue Dec 13, 2016 · 1 comment

Comments

@HiPhish
Copy link
Contributor

HiPhish commented Dec 13, 2016

I wrote a number of tests for my Info reader plugin info.vim and I noticed an odd behaviour: If I invoke Vader with a wildcard tests fail, but if I invoke the Vader file directly they pass. Here is my plugin:
https://gitlab.com/HiPhish/info.vim
https://github.com/HiPhish/info.vim (mirror)

I have tracked down the issue to the health test (health.vader). In it I have test cases that look like the following:

Execute (Correct info version number):
  let g:infoprg = 'echo 6.0'
  CheckHealth info

Then:
  let message = search('Version 6.0 of standalone info installed')
  tabclose
  AssertNotEqual 0, message

The g:infoprg variable needs to be set to the user's info binary (part of GNU Texinfo), but obviously I cannot rely on the user's system for testing :CheckHealth, so I use echo to print a fake version number (:CheckHealth causes info to be called with the --version option and parses the output for a version number).

The problem is that this variable setting persists after the test has passed. This means that all the tests in info.vader which rely on actually calling the info binary will use echo 6.0 as their binary and fail. When executing :Vader test/* the info test suite will fail, but if I execute :Vader test/info.vader it works just fine.

Is there a way to isolate test cases so that global variables (or any variables) get reset to their original values automatically? I'm using Before and After as a workaround:

Before (Store old info binary so it can be restored):
  let g:old_infoprg = g:infoprg

After (Restore the original info binary):
  let g:infoprg = g:old_infoprg

If it cannot be changed I think it would be worth adding a note under Known Issues.

@junegunn
Copy link
Owner

junegunn commented Dec 16, 2016

Vader does not track the changes of global variables, and I don't think it's possible to do that in Vim script. Cleaning up is up to the user. Using Before/After blocks or having an extra Execute block that cleans up the affected settings and variables should be the best option available.

Actually that is how I expect Vader to behave, rather than an issue. Vader executes the blocks sequentially in order, and many of my tests rely on the fact like so:

Execute (Set up vim-plug):
  call plug#begin()
  Plug 'foo'
  Plug 'bar'
  call end#()

Execute (Check g:plugs):
  " This Execute block uses the global variable set in the previous Execute block
  AssertEqual 1, len(g:plugs)

Execute (Check g:plugs_order):
  AssertEqual 1, len(g:plugs_order)

I'll think about mentioning the execution model in the README page. Thanks for the suggestion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants