Test Explorer gets stuck in infinite mode on error in ModuleInitialize

[alpaka]

Using MS Access Professional 2016 x64, I had an error in a table set up statement in ModuleInitialize inside one of my test modules. The VBA editor error window popped up, I clicked on Debug (the same happens if you click on End). After that, the Test Explorer became unresponsive (displayed the "loading" animation indefinitely) until I restarted the host app.

[ThunderFrame]

Are you able to confirm the version of Rubberduck that you're using?

Also, it sounds like VBA is executing while you're refreshing. Do you have any events triggering VBA code? Are you able to post the module/procedure that is executing when you hit the Debug button?

[alpaka]

Sorry, it's on the current version 2.0.11.2453. The code generating the error is called by the test explorer inside the ModuleInitialize procedure, I have no event code, and the error is in Access SQL where I had a syntax error, but it's irrelevant which function generates the error. The behavior is the same if I just write a mock-up with a Err.raise in it.

Public Sub RaisesError()
	Err.Raise 42, "Source", "Error"
End Sub

`Option Compare Database
Option Explicit

Option Private Module

@TestModule
Private Assert As New Rubberduck.AssertClass

'@ModuleInitialize
Public Sub ModuleInitialize()
	'this method runs once per module.
	RaisesError
End Sub

'@ModuleCleanup
Public Sub ModuleCleanup()
	'this method runs once per module.
End Sub

'@TestInitialize
Public Sub TestInitialize()
	'this method runs before every test in the module.
End Sub

'@TestCleanup
Public Sub TestCleanup()
	'this method runs after every test in the module.
End Sub

'@TestMethod
Public Sub TestMethod1() 'TODO Rename test
	On Error GoTo TestFail
	
	'Arrange:

	'Act:
	
	'Assert:
	Assert.Inconclusive

TestExit:
	Exit Sub
TestFail:
	Assert.Fail "Test raised an error: #" & Err.Number & " - " & Err.Description
End Sub

The log says

2017-01-12 11:07:58.9964;ERROR-2.0.11.2453;Rubberduck.UI.UnitTesting.TestExplorerViewModel;System.Runtime.InteropServices.COMException (0x800A9C68): Exception from HRESULT: 0x800A9C68
at Microsoft.Office.Interop.Access._Application.Run(String Procedure, Object& Arg1, Object& Arg2, Object& Arg3, Object& Arg4, Object& Arg5, Object& Arg6, Object& Arg7, Object& Arg8, Object& Arg9, Object& Arg10, Object& Arg11, Object& Arg12, Object& Arg13, Object& Arg14, Object& Arg15, Object& Arg16, Object& Arg17, Object& Arg18, Object& Arg19, Object& Arg20, Object& Arg21, Object& Arg22, Object& Arg23, Object& Arg24, Object& Arg25, Object& Arg26, Object& Arg27, Object& Arg28, Object& Arg29, Object& Arg30)
at Rubberduck.VBEditor.Application.AccessApp.Run(QualifiedMemberName qualifiedMemberName) in c:\Users\Mathieu\Documents\GitHub\Rubberduck\Rubberduck.VBEEditor\Application\AccessApp.cs:line 18
at Rubberduck.UnitTesting.TestEngine.Run(IEnumerable1 members) in c:\Users\Mathieu\Documents\GitHub\Rubberduck\RetailCoder.VBE\UnitTesting\TestEngine.cs:line 109 at Rubberduck.UnitTesting.TestEngine.Run(IEnumerable1 tests) in c:\Users\Mathieu\Documents\GitHub\Rubberduck\RetailCoder.VBE\UnitTesting\TestEngine.cs:line 75
at Rubberduck.UI.UnitTesting.TestExplorerViewModel.ExecuteRunNotExecutedTestsCommand(Object parameter) in c:\Users\Mathieu\Documents\GitHub\Rubberduck\RetailCoder.VBE\UI\UnitTesting\TestExplorerViewModel.cs:line 271
at Rubberduck.UI.Command.CommandBase.Execute(Object parameter) in c:\Users\Mathieu\Documents\GitHub\Rubberduck\RetailCoder.VBE\UI\Command\CommandBase.cs:line 54;

[ThunderFrame]

Oh, I didn't realize you were running tests, I thought the error was being raised as you were adding tests.

It seems you're missing a ' before @TestModule. Could it be that your module doesn't compile?

You also have Option Explicit and a module-level variable declared after a Sub, which isn't valid syntax.

You'll also need to make sure you've added a Reference to Rubberduck, using Tools..References, in order for the early-binding of Rubberduck.Assert to work.

Rubberduck won't work unless the VBA can be compiled.

Try refactoring so the code begins like this:

Option Explicit
Option Private Module

'@TestModule
Private Assert As New Rubberduck.AssertClass

Public Sub RaisesError()
  Err.Raise 42, "Source", "Error"
End Sub

'@ModuleInitialize
Public Sub ModuleInitialize()
  'this method runs once per module.
  RaisesError
End Sub

'....

[alpaka]

The code compiles and runs fine apart from the error when executing the ModuleInitialize. The first sub is in another module, I just copied it in for reference. Also, the missing apostrophe is due to bad copy&pasting skills on my part (haven't used Markdown in a while) ;)

I assure you that the error is not on my part.

[ThunderFrame]

Ah, so your tests aren't running because the ModuleInitialize procedure fails to complete, because your code is throwing the error?

You'll need a way to prevent the error from throwing in that procedure.

Once the error is suppressed, I suspect that Rubberduck won't know whether the procedure was successful, and will just proceed to attempt to execute the tests

We might need a way to abort tests if the ModuleInitialize method fails.

[alpaka]

That's correct, However, the actual issue (for me, at least) is not that the tests aren't running but that the Test Explorer fails to recover and becomes unresponsive.

I could of course create an error handler inside ModuleInitialize, but it would be more developer-friendly if the Test Explorer remained usable if you inadvertently create an error. The general "go to faulty line" feature of VBA is quite useful, so it'd be a shame to just swallow all errors. At the current point in time, if you introduce an error in the ModuleInitialize routine, you have to restart the host application to make Rubberduck usable again. It's not a deal-breaker but can become annoying and having completely fault-free methods here is a rather high bar for devs.

Basically, if any of the "around the test" subs fail, the Test Explorer could stop trying to execute the current test module and go to the next one.

[ThunderFrame]

hmm, maybe we need an abort tests button that is enabled while tests are running.

[retailcoder]

@ThunderFrame one problem is that RD's UI thread is waiting for the Application.Run call to return - be it with a COMException (if the VBA code threw a runtime error) or through normal execution; if the VBA code doesn't return (e.g. infinite loop), we're stuck, and the execution must be cancelled with the IDE's "stop" button (the RD thread then resumes)... so aborting a test that's running isn't possible because the main thread is already waiting. We'd need to call Application.Run on a background thread, but then the whole timing mechanics and test results compilation would fall apart.

Aborting executing VBA code that was triggered by RD isn't possible without redesigning and rewriting the whole TestEngine.

[retailcoder]

To fix this issue, all we need to do is catch any COMException that might be thrown by Application.Run... which should have been done. That's an oversight, easily fixed IMO.

[Vogel612]

Should be fixed as of 68a3d50