error on large suite with xml output #398

Turambar1 · 2014-01-31T15:26:48Z

Hi,
after upgrading to the new version, our build-process has got a problem.
Running a large suite(220 tests) on console with

java -jar fitnesse.jar -c "LargeSuite?suite&format=xml" > FitnesseReport.xml

I get an exception:

    org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getTestExecutionReport' in  class fitnesse.reporting.CachingSuiteXmlFormatter threw exception java.lang.NullPointerException at suiteXML.vm[line 6, column 38]
        at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
        at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
        at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
        at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:567)
        at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:71)
        at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:142)
        at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
        at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
        at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
        at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
        at org.apache.velocity.Template.merge(Template.java:356)
        at org.apache.velocity.Template.merge(Template.java:260)
        at fitnesse.reporting.CachingSuiteXmlFormatter.writeOutSuiteXML(CachingSuiteXmlFormatter.java:54)
        at fitnesse.reporting.CachingSuiteXmlFormatter.close(CachingSuiteXmlFormatter.java:47)
        at fitnesse.testrunner.CompositeFormatter.close(CompositeFormatter.java:29)
        at fitnesse.testrunner.MultipleTestsRunner.allTestingComplete(MultipleTestsRunner.java:57)
        at fitnesse.testrunner.MultipleTestsRunner.executeTestPages(MultipleTestsRunner.java:53)
        at fitnesse.responders.run.SuiteResponder.performExecution(SuiteResponder.java:74)
        at fitnesse.responders.run.TestResponder.doExecuteTests(TestResponder.java:81)
        at fitnesse.responders.run.TestResponder.doSending(TestResponder.java:71)
        at fitnesse.responders.ChunkingResponder.startSending(ChunkingResponder.java:66)
        at fitnesse.http.ChunkedResponse.sendTo(ChunkedResponse.java:25)
        at fitnesse.http.MockResponseSender$OutputStreamSender.doSending(MockResponseSender.java:57)
        at fitnesse.FitNesse.executeSingleCommand(FitNesse.java:90)
        at fitnesseMain.FitNesseMain.executeSingleCommand(FitNesseMain.java:94)
        at fitnesseMain.FitNesseMain.launch(FitNesseMain.java:71)
        at fitnesseMain.FitNesseMain.launchFitNesse(FitNesseMain.java:50)
        at fitnesseMain.FitNesseMain.main(FitNesseMain.java:35)
    Caused by: java.lang.NullPointerException
        at fitnesse.reporting.CachingSuiteXmlFormatter.getTestExecutionReport(CachingSuiteXmlFormatter.java:63)
        at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
        at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
        at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:173)
        ... 26 more
    org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getTestExecutionReport' in  class fitnesse.reporting.CachingSuiteXmlFormatter threw exception java.lang.NullPointerException at suiteXML.vm[line 6, column 38]
        at org.apache.velocity.runtime.parser.node.ASTMethod.handleInvocationException(ASTMethod.java:243)
        at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:187)
        at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:280)
        at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:567)
        at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:71)
        at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:142)
        at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:72)
        at org.apache.velocity.runtime.directive.Foreach.render(Foreach.java:420)
        at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:207)
        at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:342)
        at org.apache.velocity.Template.merge(Template.java:356)
        at org.apache.velocity.Template.merge(Template.java:260)
        at fitnesse.reporting.CachingSuiteXmlFormatter.writeOutSuiteXML(CachingSuiteXmlFormatter.java:54)
        at fitnesse.reporting.CachingSuiteXmlFormatter.close(CachingSuiteXmlFormatter.java:47)
        at fitnesse.reporting.BaseFormatter.errorOccurred(BaseFormatter.java:46)
        at fitnesse.responders.run.TestResponder.doExecuteTests(TestResponder.java:83)
        at fitnesse.responders.run.TestResponder.doSending(TestResponder.java:71)
        at fitnesse.responders.ChunkingResponder.startSending(ChunkingResponder.java:66)
        at fitnesse.http.ChunkedResponse.sendTo(ChunkedResponse.java:25)
        at fitnesse.http.MockResponseSender$OutputStreamSender.doSending(MockResponseSender.java:57)
        at fitnesse.FitNesse.executeSingleCommand(FitNesse.java:90)
        at fitnesseMain.FitNesseMain.executeSingleCommand(FitNesseMain.java:94)
        at fitnesseMain.FitNesseMain.launch(FitNesseMain.java:71)
        at fitnesseMain.FitNesseMain.launchFitNesse(FitNesseMain.java:50)
        at fitnesseMain.FitNesseMain.main(FitNesseMain.java:35)
    Caused by: java.lang.NullPointerException
        at fitnesse.reporting.CachingSuiteXmlFormatter.getTestExecutionReport(CachingSuiteXmlFormatter.java:63)
        at sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:395)
        at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:384)
        at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:173)
        .. 23 more

in v20130530 the output was created with out any problems.

The text was updated successfully, but these errors were encountered:

Turambar1 · 2014-01-31T15:28:38Z

The problem is that the xml output does not include the finalcounts and end up with

...
</pageHistoryLink>
          </result>
  -----Command Complete-----

Turambar1 · 2014-02-07T17:25:44Z

Hi,

Anybody can reproduce this error? Is there any hope for a fix?
We really don´t want to downgrade to an older version.

BR Tura

sergebug · 2014-02-27T23:55:53Z

I think I'm facing the same issue: One test suite consistently produces truncated XML output. I didn't see an exception yet. One difference from your situation - my output ends with closing pageHistoryLink and there is no closing result tag after it. My test suite isn't that large as yours (~80 tests) it but it takes ~20 minutes to run.

amolenaar · 2014-02-28T12:33:56Z

The xmlreportimg had changed as part of the code refactoring. Can you check
if the page history/test history is created properly? The xml formatter is
processing those files.

sergebug · 2014-02-28T16:37:52Z

There is a test result page for each of the executed tests within a testsuite page
There is no test result XML for the test suite itself.
I don't see any exceptions either.
BTW: How do I redirect logging to the flat file in the current version of FitNesse?

sergebug · 2014-02-28T18:07:42Z

Was able to resolve an error by removing completely test results and re-running test suite. Now results are published. This link gave me an idea: https://issues.jenkins-ci.org/browse/JENKINS-7168
In short: when someone renames WIKI page test results folders don't get renamed and this issue has something to do with difference in the casing: CreateAppointmentWithMedicalCaseTest vs CreateAppointmentWithMedicalcaseTest. When such thing happens FitNesse is no longer able to save test suite results as XML.

mgaertne · 2014-03-01T07:25:07Z

On which platform have you set up your environment?

sergebug · 2014-03-04T06:22:47Z

Windows as a platform. Windows Server 2008 R2 & JRE 7 to be more specific.

Golion · 2014-03-21T19:27:10Z

I'm also running into this. By trial and error I narrowed the problem down to a couple of tests that have a "!include -seamless" page pulled into a value. The important part seems to be a bang "!" inside a {{{!- -!}}} block on the seamlessly included page. (running on CentOS 5.10)

amolenaar · 2014-03-22T20:44:39Z

@sergebug: what would be the best way to fix it? I suppose it's somewhere in the file creation code. That's in the TestResponder. Does the change in amolenaar@ab3c52a help?

@Golion: Your issue is related to something different from directory names, right? I'm curious for a test case of your issue :)

sergebug · 2014-03-24T21:13:34Z

@amolenaar Given my loose understanding of issue's nature ab3c52a should suffice. I'm unable to guarantee this since issue was resolved in our environment and no longer represents itself.
We do use !include -seamless @Golion had mentioned above however we didn't change any pages in order to get valid XML output so my guess is the same as @amolenaar - different issue.
For the record: in order to "resolve" issue I did following in our environment:

Renamed files\TestResults folder into files\TestResultsSaved
Ran the culprit testsuite as XML and got output successfully. This created new files\TestResults folder
Moved content of old files\TestResultsSaved into new files\TestResults.
I didn't not get any failures ever since.

amolenaar · 2014-05-18T19:46:26Z

Should we do an extra check here when creating a test history file?

amolenaar closed this as completed in 939e420 Jun 17, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

error on large suite with xml output #398

error on large suite with xml output #398

Turambar1 commented Jan 31, 2014

Turambar1 commented Jan 31, 2014

Turambar1 commented Feb 7, 2014

sergebug commented Feb 27, 2014

amolenaar commented Feb 28, 2014

sergebug commented Feb 28, 2014

sergebug commented Feb 28, 2014

mgaertne commented Mar 1, 2014

sergebug commented Mar 4, 2014

Golion commented Mar 21, 2014

amolenaar commented Mar 22, 2014

sergebug commented Mar 24, 2014

amolenaar commented May 18, 2014

error on large suite with xml output #398

error on large suite with xml output #398

Comments

Turambar1 commented Jan 31, 2014

Turambar1 commented Jan 31, 2014

Turambar1 commented Feb 7, 2014

sergebug commented Feb 27, 2014

amolenaar commented Feb 28, 2014

sergebug commented Feb 28, 2014

sergebug commented Feb 28, 2014

mgaertne commented Mar 1, 2014

sergebug commented Mar 4, 2014

Golion commented Mar 21, 2014

amolenaar commented Mar 22, 2014

sergebug commented Mar 24, 2014

amolenaar commented May 18, 2014