Question: Is there a way to determine when tests are starting and tests are done?

[AlpsDevABE]

One of the TDD rules that I'm trying to follow is separating tests from production code.

I am leveraging the ModuleInitialize and ModuleCleanup to open and close the production (really development version) of the code that is being tested and using application.run to interface with the functions that are being tested.

The thing I'm running across is that I might have multiple testing modules testing different things, and each of them is opening and closing the production version of code. As you know each open and close comes at an IO time cost.

In this case, I would need to open the production code once before all of these tests, and then close it after all of these tests.

Is there a module that can interface before running the full test sequence like

TestSequenceInitialize
'this method runs once per test session.
TestSequenceCleanup
'this method runs once per test session.

Tangental thoughts not specifically related to question at hand, but might ask later

1. I can see a situation where maybe the production code could need to be reloaded for certain types of integration tests, but for now I would keep those cases in a different test environment for now
2. Might need to have insight on which test is running to do specific loading for certain types of test but that seems like over kill
3. A followup question I will have in the future is. Now I will have multple test files per production macro, is there a way to run tests as a full suite where each file in a test folder opens and runs test across the full sweep. This is for when one is managing many, many, many VBA files.

[retailcoder]

Rubberduck unit testing was designed with the test modules in the same project as the "production" VBA code (doesn't affect the VBA project since no execution path would load the test modules), perhaps late-bound for deployment to keep the project fully compilable.
That said having a dedicated test project to specifically not ship the unit tests along with the source code can work too, but I would recommend referencing the VBA project under test (make sure the projects are named accordingly; a project named "VBAProject" cannot reference another project also named "VBAProject") so that the project under test is always loaded when the test project is opened, and then the test project has access to all the public types and their public members (as it should, to be able to stub and setup the dependencies!), such that the objects and methods being tested are invoked directly from the tests rather than through Application.Run.

The Test Explorer UI normally updates as the tests run, showing which test is currently running (its "outcome" icon changes to a hourglass icon); there can't really be any more insight than that, ...something feels off about having a test suite that needs to know which test is currently running.

A "test suite" is essentially all test methods discovered in the IDE; there are no plans to discover test modules in a file system folder. Consider leveraging @Folder annotations in test modules, to organize the tests of a project into a custom folder hierarchy.

[AlpsDevABE]

Hey Mathieu,

Thanks for your time. I think there is one aspect of my question that wasn't directly address, and its my fault for having so may thoughts in one question.

I want to drill down this question:

1. Would there be any reason against implementing a Initilization and Cleanup procedures prior to running any tests?

Forgive my poor attempt and clarifying my ask:

Right now you guys have:

• Init/Cleanup for Test module as a whole
• Init/Cleanup for Individual Tests within a specific module

My ask would be

• Init/Cleanup for running any tests 'as a session'

Session being any of the following:

• Run => All Tests
• Run => Individual test
• Run => Past Test
• etc...

So for example if I had three testing modules with two tests each, clicking the Run All Tests would kick off a process that looks like this:

Init Current Test Session // The Ask

    Init TestModule1

        Init TestModule1.Test1
            TestModule1.Test1
        Cleanup TestModule1.Test1

        Init TestModule1.Test2
            TestModule1.Test2
        Cleanup TestModule1.Test2

    Cleanup TestModule1

    Init TestModule2

        Init TestModule2.Test1
            TestModule2.Test1
        Cleanup TestModule2.Test1

        Init TestModule2.Test2
            TestModule2.Test2
        Cleanup TestModule2.Test2

    Cleanup TestModule2

Cleanup Current Test Session // The Ask

[SmileyFtW]

Just thinking out loud... Seems to me that since there is no overhead other than file size to retaining the Test Modules in the production file, it would be valuable to have the test modules within the file it tests; that way if/when someone else (or you some time in the future once you've lost, deleted, or misplaced the separate test file) needs to update or troubleshoot the code the testing is ready and available. As I have come to learn, the test methods are very good documentation of what the code is expected to do.

[AlpsDevABE]

Hey guys,
I waited a few days to see if anything else would add to this conversation but it seems to have run dry. I will go ahead and close this ticket as I seem to understand where you guys are approaching this project from.

Thanks for the awesome project, I use it everyday and was successfully able to set up a Test Driven Environment using this platform.

To share some knowledge, Ultimately the decision really is to separate production code from tests. This is one of the "accepted practices" in writing tests from all the research that I've made. [Rightly or wrongly]

What I've done is I wrote two supporting libraries for any test file we have.

One file takes care of opening and closing workbooks.
All one needs to do is pass the filepath and password to a module to open, and pass the workbook to close to close them.

The way the module works is that it keeps track of all the files it has open, and it will only close the files it has opened itself. It looks to see if a file is already open prior to opening it and grabs that instead of attempting to open.

This has the following advantages:

• If i'm just running tests, then the file I am working with will open, run tests, and then close
• If I opened the file I'm working with to code something, the tests will run fine, but will not close my file
• Current challenge remaining: How to not open and close the file for each test module

The second library fully extends out the Application.Run

This is in order to make interfacing with the workbook being tested easier. Pass in the workbook the function to be called, return type and any additional parameters and it will take care of running that procedure on its own.

I can write functions that interface with the function being called and put those in a module and it leverages the library to do the actual calls.

That seems to be a good starting point for what we are doing. I'm aware that I could add the production code as a reference, but doing so makes it harder to run the type of tests that need to close that workbook and reopen it. There are a few of those, and that could get annoying in our environment.

I think that is the best solution for us, and feel free to ask me more if you think this is a good setup.

One of the wanted features would be to open all the test files and run the tests in an automated way and get the results. If there was a VBA plugin of some sort that could access the rubber duck libarary in order to do those commands that would be a good way to move forward. This might already exist but I haven't looked into this.

[retailcoder]

This is one of the "accepted practices" in writing tests from all the research that I've made. [Rightly or wrongly]

That would be correct, but with VBA not having a "solution" level that "contains" multiple projects, it's much simpler to keep the tests in the same project; since there's never been a first-party unit testing framework for VBA, VBA-specific guidance is pretty much inexistent. If you browse through Rubberduck's source code, you see that indeed we do have all the tests in their own separate dedicated project - but the solution consists of more than a dozen projects and whenever we load Rubberduck in Visual Studio we always have the tests loaded alongside the rest of the projects.

That said there's nothing wrong with keeping the tests as a separate VBA project referencing the project being tested - except that would not work in all host applications (many don't support loading multiple projects at the same time), so it's unlikely that the engine will be modified to support a relatively fringe scenario that cannot work across all the applications Rubberduck can be hosted in.

If there was a VBA plugin of some sort that could access the rubber duck libarary in order to do those commands that would be a good way to move forward.

Indeed! However there doesn't appear to be an available command-line argument to start a host and fire up the VBE so it can load its add-ins; getting VBA unit tests to run unattended/scripted unfortunately doesn't appear to be a possibility in any VBA host application. Perhaps something written in VBA involving the VBIDE Extensibility library could do something like that, but that would require lowering security settings in the host, ..there's a balance to hit between usability and macro security there: we wouldn't want malware to start targeting machines with Rubberduck installed because these would necessarily allow executing code that accesses the VBIDE API!

[AlpsDevABE]

Thanks for filling in the gaps in my knowledge for me. :)

[SmileyFtW]

The thing I like about having the tests in the VBA Project is that someone else picking up a project can look at the tests and see what the item under test is supposed to do and find edge cases or add new capability - knowing just where that new capability might best be implemented... IOW self-documenting code.