Add a unit testing framework.

[metakirby5]

Really, this should have been here from the start, but I was lazy 😛 I need to look into this a bit, but in the meantime, please chime in with any suggestions.

[balta2ar]

Another framework to consider is https://github.com/thinca/vim-themis.

[balta2ar]

One more candidate, just found it today: https://github.com/kana/vim-vspec

[metakirby5]

I've been thinking about this for a bit, and I don't know if either framework can satisfy the requirements of testing Codi, namely:

• Test both Vim and NVim in one go
• Test using both BSD and Linux versions of script
• Support asynchronous assertions (i.e. fill buffer with code and assert the contents of the Codi buffer on CodiUpdate)

I might just have to roll a custom solution for this. Thoughts?

[tjdevries]

Personally I like this. You might be able to get it to work for you.

Might have to do some work with setup to get vim and nvim working together? not sure.

https://github.com/junegunn/vader.vim

[still-dreaming-1]

Perhaps this is already understood or missing the point, but something like Travis CI might be helpful. I believe it creates a fresh VM for each test and installs different software and then runs tests on it. So you could have it install Vim, Neovim, different versions of script, etc. You may want to use that in conjunction with an existing Vim testing framework.

It also might be helpful to break up your tests into feature tests and unit tests. For true unit tests, I like to break up my VimL code into "classes" so I can do OO and then test the classes with vim-UT. vim-UT feels like it is treating VimL as a programming language rather than just providing a way to check buffer contents and stuff like that. It is good for testing the basic logic and algorithms of your VimL code.

You can create "classes" in VimL by having factory functions inside of files in the autoload directory that return dictionary variables. The dictionary variables act as "objects". You can attach "methods" to them. Here are some examples: https://github.com/still-dreaming-1/vim-elhiv/tree/master/autoload You will probably want to be more responsible than me and prepend you class file names with codi_ instead of l_.

[metakirby5]

@tjdevries I also wanted to give vader.vim a shot, but I wasn't sure if it would support async in tests. However, I haven't looked all that deeply into it, so I'll try it out.

@still-dreaming-1 Interesting, I wasn't aware Vim had OO support. Breaking functionality up into logical pieces will certainly help with overall testability. I also knew of Travis CI but haven't had extensive experience with it, so I'll definitely look into it some more.

Thank you all for the thoughtful suggestions :) Hopefully we can transform this little pile of hacked together VimL into something dependable.

[still-dreaming-1]

Now if there was a good VimL REPL, then I could definitely see making your own testing framework being a good thing. This is because you could design that framework in such a way that the code that appears in the test files could be entered into a REPL and have that in itself run the tests and show the feedback from failing assertions. Actually I think testing frameworks that are designed to run that way are the killer app of codi. Otherwise it is hard to test functions and classes with code. This is not a limitation of codi, but of REPLs. Let me see if I can explain... When you experience codi, you want to get feedback from all the code you write. But then when you are writing a function, suddenly you are only getting limited feedback. It only tells you when you are breaking the rules of the language enough to cause an error, rather than showing the results of algorithms. It works that way for 2 good reasons. 1) There is no way for the REPL to know what values you will pass into the function. 2) The existence of a function does not mean you intend for the code to run and do anything yet, it is just there waiting to be called, so it would be a bug in the REPL if the function was called and run just from defining it.

Now the obvious way to get around this is have some code just after the function that exercises it. But then you just have a one-off test that will get thrown away when you are happy with it. It is nice to have that option, but somewhat limited. Now if your unit tests could be run in REPL by codi, this would get around that problem. Unit tests are designed to pass in a bunch of values and exercise all the paths, use cases, and edge cases of the code and make sure the result is what was expected. The problem with trying to start codi from a unit tests file, is that typically the actual code that exercises your other code is also inside of functions or methods, so codi does not run them. However it is possible to design testing frameworks so that the code setting up and running the assertions exists outside of any functions or classes. I believe those testing frameworks are the killer app of codi. If you could make one for VimL, then you could also use codi to test codi, which would be way nicer than using something else to test codi...

Anyway trying to actually make a REPL for Vim or turn Vim into a REPL is probably not a good idea as it would probably be more of a huge distraction to testing codi, but I stand by what I said that those types of testing frameworks are the killer apps of codi. As I mentioned once before, I am making one for Node.js called living-tests. I just recently got it working and am using it to test one of my node projects.

[metakirby5]

@still-dreaming-1 living-tests is pretty cool! While I don't think I should be using Codi to be testing Codi (as in that case, one bug can cause completely unrelated tests to fail), I think the instant feedback test framework idea has a lot of merit.

[balta2ar]

Here is another one: https://github.com/google/vroom