Combine existing testresult.xml file

[CharliePoole]

@georgooty commented on Wed Aug 16 2017

Hi,

Have any option to combine the test results.xml with existing test results xml file.

Regards,
George

---

@CharliePoole commented on Thu Aug 17 2017

No, but it's an interesting idea and the code to do it is already present in the engine.

[ChrisMaddock]

Is this a feature we'd want to add to the core console runner?

A suggestion - if #148 was implemented, this could then be done entirely within a IResultWriter extension, could it not?

My opinion is that this shouldn't part of the core program, but is a prime feature that our extensibility options should support.

[mikkelbu]

If it can be done within an extension then I agree with @ChrisMaddock

[rprouse]

What is the scenario? Is this intended to combine the current test run with the existing results from previous test runs or is it intended to combine the results from multiple simultaneous test runs? The first doesn't seem very useful to me and the second requires coordination between runners.

Would a small app that uses the result writer to join result files be more flexible?

[CharliePoole]

There's no feature in the engine where we load a result file, much less several. And the result writer interface doesn't have a suitable method to implement. At least that's the case as I understand the idea.

Maybe I'm missing something @ChrisMaddock - can you outline how you see this working?

[CharliePoole]

@rprouse Your comment just popped up. I'm assuming the second scenario. It could be another app or a new top level option in the console runner like explore or run.

I still don't see the relevance of a result writer extension since we aren't talking about a new format.

The code in the engine that combines multiple XML results is internal code. It could be exposed or just copied to a new app.

[ChrisMaddock]

To be clear - my current opinion is that this doesn't belong in the console runner. @CharliePoole - you marked it as an idea rather than closing, so I was looking at workarounds. 🙂

Maybe I'm missing something @ChrisMaddock - can you outline how you see this working?

If the user could pass in a custom flag, and access that value in an extension, then a user could implement all this functionality in their own IResultWriter extension. e.g.

1. User passes in --combinewith=oldresult.xml.
2. Console runs tests as normal
3. When IResultWriter extension is hit - the user's extension...
   a. Loads/Parses oldresult.xml
   b. Takes the current TestEngineResult objects, and combines it with oldresult.xml
   c. Writes out a result file that combines both files.

As far as I can see, this would all be possible to be implemented outside of NUnit, if the user had a way to pass in the file they wished to combine with (#148).

[CharliePoole]

@ChrisMaddock With feature requests, I no longer close anything. I use is:idea if I feel we might not want to do it.

Nevertheless, this interests me because it's a potential engine feature I'd make use of in the gui.

My interpretation of the idea, however, was not the same as yours. I was thinking we would open multiple XML files on the command line.

Regarding doing anything like this in an extension, that implies either duplicating engine code in the extension or giving extensions a reference to the engine and exposing the code that combines results, perhaps as a service.

[ChrisMaddock]

What would be the use with the GUI? 🙂

[CharliePoole]

GUI use cases...

1. Open a test result and review it.

2. Open a set of assemblies with prior results associated and display them.

3. Do (2) and then rerun.

4. Do any of the above and save the combined result.

As the gui progresses, I expect to do PRs on the engine as needed.

[rprouse]

@ChrisMaddock I wouldn't take this too far at this point. We don't really have the resources to start a new project or app and we still don't have a specific use case for this. @CharliePoole has outlined a few, but even those are different than my original use cases.

To be honest, our XML format is known, it would be pretty easy for someone to write a utility that used LINQ to XML to read in the results and combine them in a way that is useful to them. Without specific use cases, I think we would have more trouble satisfying the needs of many users.

[ChrisMaddock]

I'm going to close this one now.

I think this idea in itself is out of scope for the engine/console, mainly due to the low interest. I'd love for this to be possible to implement as an extension - our extensibility mechanisms need some work to make that happen, but I think we have separate issues tracking those ideas.