Feature Request: Run each spec in a separate process

[patrickwcummings]

We use Gauge to automate a Java desktop application that can only support one running instance on a given system. Our Gauge steps actually launch the application via a dependent JAR and interact with the application using Java APIs.

This works fine when we run one spec at a time, but due to all specs running in the same runner process, it does not work when running multiple specs. There's no way to terminate the application that we have launched in Gauge. We explored several options:

1. Have the application issue a System.exit() when a spec is done (by closing the application). Unfortunately, this kills the gauge process as well, so we get no reports and subsequent specs don't run.

2. Start the application in a separate process. But this would require coming up with a communication mechanism between the Gauge steps and the application's separate process.

3. Have an option for gauge core to run each spec in a separate runner process. The application goes away when this process ends, which allows us to restart the application in subsequent specs.

I have created a local version of Gauge that does just that, so I can submit a PR if this is something you're willing to introduce into Gauge. I created a new flag --isolated that would use a new execution type (instead of parallelExecution or simpleExecution).

I'm also open to other suggestions to accomplish the same thing.

Gauge version

1.0.5

[sriv]

Our Gauge steps actually launch the application via a dependent JAR

seems to suggest that your test is launching the application-to-be-tested inProcess? i.e. your tests and the application to be tested are the same process? Is this correct?

Have an option for gauge core to run each spec in a separate runner process. The application goes away when this process ends, which allows us to restart the application in subsequent specs.

I am guessing here that you have enable_multithreading set to true. If you set this to faise, gauge should use a separate process for each stream when run in parallel and each runner executes one spec at a time. Note that the runner is not recycled, and hence this isn't exactly what you request - 'one process per spec'.

Is this acceptable?

[patrickwcummings]

Thanks for your response. I forgot to mention that the first thing we tried was to use parallel runs, but unfortunately that didn't work for our case either.

seems to suggest that your test is launching the application-to-be-tested inProcess? i.e. your tests and the application to be tested are the same process? Is this correct?

That's correct. It's a desktop application that doesn't have a means to drive it from a separate process, so we launch it via the tests in the same process.

I am guessing here that you have enable_multithreading set to true. If you set this to faise, gauge should use a separate process for each stream when run in parallel and each runner executes one spec at a time. Note that the runner is not recycled, and hence this isn't exactly what you request - 'one process per spec'.

I did not have multithreading set to true (though it's good to know it exists - I wasn't aware of that). The problem is as you mentioned - that the runner isn't recycled.

I don't know how often Gauge is used to test a desktop application. It works really well, but I haven't found a way to terminate the application without killing Gauge itself. Killing the runner does terminate the application.

We did entertain lifting the requirement that only one instance of our application can run at a time in automation, but that causes other problems. Either we launch numerous instances of the application if each spec launches one, which causes machine resource and Java static variable issues, or each spec needs to leave the application in a state that works for the subsequent spec, which is difficult to maintain.

[sriv]

Thanks for the additional information. It helps.

I guess we are in agreement that launching too many instances of the application will drain the system resources. In fact, we added the multithreading ability to allow users to avoid jvm overhead for every parallel stream.

I don't know how often Gauge is used to test a desktop application

I reckon not a lot! You are probably the first person in the community to have declared it. Thank you, it helps a lot when we know what sort of use cases our users seek and envision.

haven't found a way to terminate the application without killing Gauge itself. Killing the runner does terminate the application.

Have you considered a simple launcher app/script that can trap a kill request and gracefully shutdown the application and bring it back up when needed?

I still get a feeling that this feature could not be Gauge's concern, because it is more to do with the way the application is run and shutdown rather than it being tested.

[patrickwcummings]

Have you considered a simple launcher app/script that can trap a kill request and gracefully shutdown the application and bring it back up when needed?

That's basically what we do. We actually trap the kill request at this point and do nothing with it. The root problem is that I don't think there's any way to gracefully shut down the application. Because the test process and the application's process are one and the same, even gracefully shutting down the application ends up killing Gauge. "Gracefully shutting down" could mean, I suppose, simply hiding the application window and doing some cleanup, but again, then we have to make sure we clean up everything and get the application back to an initial state. Easier said than done, and involves application code that is now different under test.

It's possible I didn't understand what your thoughts were regarding gracefully shutting down. Did you have something else in mind?

I still get a feeling that this feature could not be Gauge's concern, because it is more to do with the way the application is run and shutdown rather than it being tested.

I do understand and appreciate this viewpoint. In thinking about this more, I think anybody trying to automate a desktop application would run into a similar problem, unless they have a way to drive the application running as a separate process. Perhaps rather than thinking of it solely as an "isolated process" feature, maybe it's more of a "desktop mode"?

[sriv]

Because the test process and the application's process are one and the same

I was wondering if this could be split? i.e. have a wrapper java project that is executable, which can include all the dependencies that your application may have. and on launching this, it could bring up your application. Gauge tests can invoke this wrapper as a sub process/ sub shell and run tests.

This way you could perhaps kill the application without killing gauge-java (or gauge). Or am I missing something here?

[sriv]

I assume your application is not publicly shareable, but if you have a sample app that mimics the behaviour of your application's launching, that'll help troubleshoot better.

[patrickwcummings]

I assume your application is not publicly shareable, but if you have a sample app that mimics the behaviour of your application's launching, that'll help troubleshoot better.

While you're correct that our application isn't publicly shareable, I can definitely create a smaller app that mimics it. It will take me a little time to do it - I'm out of the office for most of this week.

I was wondering if this could be split?

Yes, it can be split, and I'm sure I could launch and terminate the application in that case without issue. The issue then is driving the launched application via the tests. We're basically using the Java Robot API (via TestFX) to drive the application, which is what is forcing the tests and the application to be the same process. The wrapper application would have to somehow forward these requests on to the application.

In any case, your idea of sharing the application is a good one. I'll start working on that.

[zabil]

Closing this as Gauge will not be implementing this feature.