BIRT POJO runtime not working

[hvbtup]

I used the Eclipse setup to download and compile the current "master" today.

This generated the file birt-runtime-4.9.0-20220105.zip (in build\birt-packages\birt-runtime\target).
Unzipped this file and added \Path\to\ReportEngine\lib* to the CLASSPATH environment variable.
Then I tested creating reports from within our application.

This fails with an exception like

org.eclipse.birt.report.engine.api.EngineException: Error happened while running the report.
at org.eclipse.birt.report.engine.api.impl.EngineTask.handleFatalExceptions(EngineTask.java:1826)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:149)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run(RunAndRenderTask.java:65)
at com.tp.server.lisa.birt.report.BirtReportGenerator.run(BirtReportGenerator.java:433)
Caused by: java.lang.ExceptionInInitializerError
at org.eclipse.birt.report.engine.emitter.pdf.PDFRender.createPageDevice(PDFRender.java:62)
at org.eclipse.birt.report.engine.layout.emitter.PageDeviceRender.start(PageDeviceRender.java:111)
at org.eclipse.birt.report.engine.layout.emitter.PageEmitter.start(PageEmitter.java:50)
at org.eclipse.birt.report.engine.nLayout.LayoutEngine.start(LayoutEngine.java:265)
at org.eclipse.birt.report.engine.emitter.CompositeContentEmitter.start(CompositeContentEmitter.java:171)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:140)
... 2 more
Caused by: java.lang.NullPointerException
at org.eclipse.core.internal.runtime.InternalPlatform.getBundles(InternalPlatform.java:232)
at org.eclipse.core.internal.runtime.InternalPlatform.getBundle(InternalPlatform.java:222)
at org.eclipse.core.runtime.Platform.getBundle(Platform.java:1442)
at org.eclipse.birt.report.engine.internal.util.BundleVersionUtil.getBundleVersion(BundleVersionUtil.java:33)
at org.eclipse.birt.report.engine.emitter.pdf.PDFPageDevice.(PDFPageDevice.java:84)
... 8 more

Another report (which uses the DOCX emitter instead of the PDF emitter failed with a similar error in the stacktrace while trying to load the JDBC driver.

Has anybody successfully tested the (non-osgi) runtime?

Note:

Generating reports worked with Java 8 and my fork hvbtup/birt (branch flugtiger-master).

I've performed the process of building BIRT (using Maven) and using the generated birt-runtime-4.*.zip several times and never had any problems until now, thus I am pretty sure that my environment variables etc. are ok.
The number and size of the jar files in reportengine/lib seems to be ok as well.

The difference is that now I am using the eclipse/master trunk (after the "BIRT reboot") and Java 11 instead of Java 8.

In case of PDFPageDevice.java we know exactly what bundle is searched, because the error happens in this line:

	protected static String[] versionInfo = new String[] {
			BundleVersionUtil.getBundleVersion("org.eclipse.birt.report.engine") };

As a side note, launching "BIRT all-in-one.launch" and previewing reports from the launched "Report-All-In-One" designer works like a charm.
The only thing I had to adjust to make it work was to check an entry for OSGI in the .launch file:

Can someone please confirm that the BIRT runtime (generated fresh from master) is working with Java 11 or tell me what's wrong here?

[wimjongman]

Can you share a simple report and a test class that demonstrates the issue? I don't know how to start the runtime at the moment. I did it a few years back but you would save me a lot of time if you can attach a simple project that demonstrates the issue.

[hvbtup]

The simplest test of the POJO runtime that I can imagine is the following:

Preparation: Compile and "install"

Use MVN package to compile and package everything.

You'll find a ZIP file birt-runtime-4.9.0-2022*.zip in build\birt-packages\birt-runtime\target

Unzip this to a new directory, say c:\runtime-test.

You now have a structure like this:

Preparation: Fix the generated script

Use a text editor to fix the file C:\runtime-test\ReportEngine\genReport.bat:

These lines ...

REM set the birt class path.
setlocal enabledelayedexpansion
set BIRTCLASSPATH=
for %%i in (%WORK_DIR%\lib\*.jar) do set BIRTCLASSPATH=%%i;!BIRTCLASSPATH!

... must be replaced with this:

set BIRTCLASSPATH=%WORK_DIR%\lib\*

A few lines below, change this line ...

SET JAVACMD=java

to something like ...

SET JAVACMD=C:\prog\jdk\jdk-11.0.13+8\bin\java.exe

to make sure that Java 11 and no other Java version is used. Of course you need to adapt this to where your Java 11 is.

Test the POJO runtime

Use the command line (cmd.exe, not PowerShell) and CD into the ReportEngine directory, then enter this command:

genReport.bat -m runrender -f PDF samples\hello_world.rptdesign

And you'll get a wonderful stack trace:

etc. ending with these lines:

Caused by: java.lang.NullPointerException
at org.eclipse.core.internal.runtime.InternalPlatform.getBundles(InternalPlatform.java:232)
at org.eclipse.core.internal.runtime.InternalPlatform.getBundle(InternalPlatform.java:222)
at org.eclipse.core.runtime.Platform.getBundle(Platform.java:1442)
at org.eclipse.birt.report.engine.internal.util.BundleVersionUtil.getBundleVersion(BundleVersionUtil.java:33)
at org.eclipse.birt.report.engine.emitter.pdf.PDFPageDevice.(PDFPageDevice.java:84)
... 10 more

[hvbtup]

I didn't really look at the code, but I believe the error is caused by some code change deep inside the Eclipse core, not inside BIRT. Or perhaps it may be related to the newer Java release.

[wimjongman]

Thanks, that is very helpful. I will take a look.

[hvbtup]

@wimjongman John Ward said on the birt-dev list that he wanted to take a look at it, too. Unfortunately I don't know his GitHub account. I'd be interested if he has made any progress yet.

For the PDFPAgeDevice.java error, it is interesting to note that the error happens in static initialization code.

AFAIK static variables are considered bad practice with OSGI !?

So maybe this can be fixed by somehow avoiding this static initialization.

But I think this is not the whole truth, because when I am using the Word emitter in out application, I get a similar error later, when BIRT tries to load the JDBC driver (which is contained in the class-path).

I'm not so much a Java insider (the times when I used Java every day were back in the last century), but it seems to me this regression is caused by some changes in the class loading mechanism, either between Java 8 and 11 or between the older and newer Eclipse core framework.

By chance, I found a message here https://www.eclipse.org/forums/index.php/t/1104321/ where Sascha Müller said that he had built BIRT and made it work in June 2020 - and someone else replied that it didn't work for him or her - with exactly this ExceptionInInitalizationError in PDFPageDevice.java! Probably Sascha's Eclipse doesn't use the POJO runtime.

I forked hvbtup/birt from flugtiger/birt (branch flugtiger_master) before October 2019, then I merged from flugtiger/birt on May 22, 2020. The POJO runtime of my fork is working fine in production use.

This make me think that this issue is caused by some changes between flugtiger/birt (flugtiger_master) as of 2020-05-22 and the code Sascha Müller used for https://github.com/SaMuellerDLR/birt (he also forked from flugtiger/birt).
These changes could be inside the BIRT source code or caused by newer releases of the plugins, e.g. the Eclipse core.

[wimjongman]

Henning, I just found some time to look at this. The good news is that I have found the issue, the bad news is that it was all too simple and I'm sorry you had to spend that much time and frustration.

Instead of using the birt-runtime, please use the birt-runtime-osgi. [1]

[1a]

• https://ci.eclipse.org/birt/job/birt-master/lastSuccessfulBuild/artifact/build/birt-packages/birt-runtime-osgi/target/

[1b]

See the artifacts of a recent action run. One of https://github.com/eclipse/birt/actions

• e.g.https://github.com/eclipse/birt/actions/runs/1736215688

[wimjongman]

[hvbtup]

@wimjongman Can you tell me what caused the issue?

I can confirm your test, it is working with the osgi-runtime from the lastSuccessfulBuild.

What causes me headache is the fact that I built the OSGI runtime 14 days ago and then it didn't work (see my comment in #738) - and I can confirm that result, too.

So what has changed in the last 14 days?

I'll compare the generated ReportEngine directories of both builds...

[hvbtup]

BTW with Java 17 at least the exception (when using the POJO runtime) is clearer:

Caused by: java.lang.NullPointerException: Cannot invoke "org.osgi.framework.wiring.FrameworkWiring.findProviders(org.osgi.resource.Requirement)" because "this.fwkWiring" is null
at org.eclipse.core.internal.runtime.InternalPlatform.getBundles(InternalPlatform.java:232)
at org.eclipse.core.internal.runtime.InternalPlatform.getBundle(InternalPlatform.java:222)
at org.eclipse.core.runtime.Platform.getBundle(Platform.java:1442)
at org.eclipse.birt.report.engine.internal.util.BundleVersionUtil.getBundleVersion(BundleVersionUtil.java:36)
at org.eclipse.birt.report.engine.emitter.pdf.PDFPageDevice.(PDFPageDevice.java:84)
... 10 more

My comparison of the builds didn't give me more insight, unfortunately.
I can see that the files are more or less the same (some differences caused by timestamps in the filenames)
There are some differences in some xml and properties files inside JAR files.
A lot of differences in class files, but that doesn't necessarily mean that the code is actually different.
I wish we had reproducible builds.

[hvbtup]

It is worth noting that using the OSGI runtime in production is a bit more complicated than the POJO runtime.

• You have to make a tiny change in your code (see JDBC issue with OSGi runtime #787)
• The disk (and maybe memory) footprint is significantly bigger.
• The OSGI runtime creates log files and lots of other files in ReportEngine\platform\configuration and who knows where else?
• E.g. ReportEngine\platform\configuration\org.eclipse.core.runtime.manager contains several .tmp*.instance files.
  So this definitely needs additional house-keeping work.
• Lots of error messages on stderr on startup (see JDBC issue with OSGi runtime #787)

To summarize, replacing the POJO runtime with the OSGI runtime is definitely possible, but not quite straightforward.

[wimjongman]

The POJO should run as well. I have to take a look at that NPE. It should not be too difficult.

My plan is to see if that class that returns an NPE was changed in eclipse and if it is a regression that stops us. However, I have to find some time.

[patric-r]

+1 for keeping the POJO runtime

[wimjongman]

Ok, the latest commit fixed the POJO runtime. I have to still polish it but you can take the latest build for a spin. e.g. https://github.com/eclipse/birt/actions/runs/1742145831

[hvbtup]

Didn't yet test your commit, I still try to make our app working with the OSGI runtime. An additional problem might arise if an application uses the BIRT OSGI runtime from several processes (like we do).
I tried specifying system properties to make each BIRT process use a separate directory:

-DBIRT_HOME=E:\reporting\jlib\birt\v490_osgi_20220123\ReportEngine\platform
-Dosgi.locking=none
-Djava.io.tmpdir=E:\reporting\temp\birt-2
-Declipse.p2.data.area=E:\reporting\temp\birt-2\p2
-Dosgi.configuration.area=E:\reporting\temp\birt-2\configuration
-Dosgi.instance.area=E:\reporting\temp\birt-2\workspace

(I googled for these properties)

Before I start the BIRT process, I created the E:\reporting\temp\birt-2 directory and copied the structure from birt-runtime-osgi-4.9.0-20220123.zip\ReportEngine\platform\configuration to E:\reporting\temp\birt-2\configuration.

In the process, I use the BIRT_HOME property to call engineConfig.setEngineHome.

Despite the osgi.configuration.area setting, BIRT creates .manager directories with *.instance files under E:\reporting\jlib\birt\v490_osgi_20220123\ReportEngine\platform\configuration.

This makes me fear issues when several BIRT processes are running simultaneously.

[hvbtup]

@wimjongman Now I tested your POJO runtime fix (downloaded from the link you specified).

Unfortunately that didn't work - it failed to load the Oracle JDBC driver.

org.eclipse.birt.report.engine.api.EngineException: An exception occurred during processing. Please see the following message for details:
The driver org.eclipse.birt.report.data.oda.jdbc's initial entry point cannot be found or cannot be instantiated.
org.eclipse.datatools.connectivity.oda.consumer.helper.OdaHelperException: Unable to load or instantiate the custom ODA driver class. (org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver) ;
java.lang.NullPointerException
at org.eclipse.birt.report.engine.executor.ExecutionContext.addException(ExecutionContext.java:1060)
at org.eclipse.birt.report.engine.executor.ExecutionContext.addException(ExecutionContext.java:1045)
at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:84)
at org.eclipse.birt.report.engine.executor.GridItemExecutor.execute(GridItemExecutor.java:63)
at org.eclipse.birt.report.engine.executor.ReportExecutorUtil.executeAll(ReportExecutorUtil.java:73)
at org.eclipse.birt.report.engine.executor.ReportExecutorUtil.executeAll(ReportExecutorUtil.java:77)
at org.eclipse.birt.report.engine.executor.ReportExecutor.createPageExecutor(ReportExecutor.java:198)
at org.eclipse.birt.report.engine.internal.executor.dup.SuppressDuplciateReportExecutor.createPageExecutor(SuppressDuplciateReportExecutor.java:57)
at org.eclipse.birt.report.engine.internal.executor.wrap.WrappedReportExecutor.createPageExecutor(WrappedReportExecutor.java:44)
at org.eclipse.birt.report.engine.executor.ReportExecutorUtil.executeMasterPage(ReportExecutorUtil.java:56)
at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.start(HTMLPageLM.java:132)
at org.eclipse.birt.report.engine.layout.html.HTMLPageLM.layout(HTMLPageLM.java:85)
at org.eclipse.birt.report.engine.layout.html.HTMLReportLayoutEngine.layout(HTMLReportLayoutEngine.java:96)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.doRun(RunAndRenderTask.java:144)
at org.eclipse.birt.report.engine.api.impl.RunAndRenderTask.run(RunAndRenderTask.java:68)
at com.tp.server.lisa.birt.report.BirtReportGenerator.run(BirtReportGenerator.java:433)
Caused by: org.eclipse.birt.report.data.adapter.api.AdapterException: An exception occurred during processing. Please see the following message for details:
The driver org.eclipse.birt.report.data.oda.jdbc's initial entry point cannot be found or cannot be instantiated.
org.eclipse.datatools.connectivity.oda.consumer.helper.OdaHelperException: Unable to load or instantiate the custom ODA driver class. (org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver) ;
java.lang.NullPointerException
at org.eclipse.birt.report.data.adapter.impl.DataRequestSessionImpl.execute(DataRequestSessionImpl.java:585)
at org.eclipse.birt.report.engine.data.dte.DteDataEngine.doExecuteQuery(DteDataEngine.java:137)
at org.eclipse.birt.report.engine.data.dte.AbstractDataEngine.execute(AbstractDataEngine.java:249)
at org.eclipse.birt.report.engine.executor.ExecutionContext.executeQuery(ExecutionContext.java:1628)
at org.eclipse.birt.report.engine.executor.QueryItemExecutor.executeQuery(QueryItemExecutor.java:73)
... 13 more
Caused by: org.eclipse.birt.data.engine.odaconsumer.OdaDataException: The driver org.eclipse.birt.report.data.oda.jdbc's initial entry point cannot be found or cannot be instantiated.
org.eclipse.datatools.connectivity.oda.consumer.helper.OdaHelperException: Unable to load or instantiate the custom ODA driver class. (org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver) ;
java.lang.NullPointerException
at org.eclipse.birt.data.engine.odaconsumer.ExceptionHandler.newException(ExceptionHandler.java:50)
at org.eclipse.birt.data.engine.odaconsumer.Driver.createNewDriverHelper(Driver.java:94)
at org.eclipse.birt.data.engine.odaconsumer.DriverManager.getDriverHelper(DriverManager.java:101)
at org.eclipse.birt.data.engine.odaconsumer.DriverManager.getNewDriverHelper(DriverManager.java:91)
at org.eclipse.birt.data.engine.odaconsumer.ConnectionManager.openConnection(ConnectionManager.java:138)
at org.eclipse.birt.data.engine.executor.DataSource.newConnection(DataSource.java:201)
at org.eclipse.birt.data.engine.executor.DataSource.open(DataSource.java:190)
at org.eclipse.birt.data.engine.impl.DataSourceRuntime.openOdiDataSource(DataSourceRuntime.java:191)
at org.eclipse.birt.data.engine.impl.QueryExecutor.openDataSource(QueryExecutor.java:412)
at org.eclipse.birt.data.engine.impl.QueryExecutor.prepareExecution(QueryExecutor.java:297)
at org.eclipse.birt.data.engine.impl.PreparedQuery.doPrepare(PreparedQuery.java:378)
at org.eclipse.birt.data.engine.impl.PreparedDataSourceQuery.produceQueryResults(PreparedDataSourceQuery.java:156)
at org.eclipse.birt.data.engine.impl.PreparedDataSourceQuery.execute(PreparedDataSourceQuery.java:145)
at org.eclipse.birt.data.engine.impl.PreparedOdaDSQuery.execute(PreparedOdaDSQuery.java:160)
at org.eclipse.birt.report.data.adapter.impl.DataRequestSessionImpl.execute(DataRequestSessionImpl.java:572)
... 17 more
Caused by: org.eclipse.datatools.connectivity.oda.consumer.helper.OdaHelperException: Unable to load or instantiate the custom ODA driver class. (org.eclipse.birt.report.data.oda.jdbc.OdaJdbcDriver) ;
java.lang.NullPointerException
at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaDriver.loadDriverInstance(OdaDriver.java:251)
at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaDriver.init(OdaDriver.java:198)
at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaDriver.(OdaDriver.java:113)
at org.eclipse.birt.data.engine.odaconsumer.Driver.createNewDriverHelper(Driver.java:91)
... 30 more
Caused by: java.lang.NullPointerException
at org.eclipse.core.internal.runtime.InternalPlatform.getBundles(InternalPlatform.java:232)
at org.eclipse.core.internal.runtime.InternalPlatform.getBundle(InternalPlatform.java:222)
at org.eclipse.core.runtime.Platform.getBundle(Platform.java:1442)
at org.eclipse.datatools.connectivity.oda.consumer.helper.DriverExtensionHelper.loadDriverClass(DriverExtensionHelper.java:125)
at org.eclipse.datatools.connectivity.oda.consumer.helper.OdaDriver.loadDriverInstance(OdaDriver.java:224)
... 33 more

As shown in the stack trace, it seems that your fix does not include the calls to Platform.getBundle from insided the datatools - just a wild guess.

[wimjongman]

Yes, that is correct. That is a different project altogether. I try to fix that in platform.

[wimjongman]

I have fixed it upstream.

https://bugs.eclipse.org/bugs/show_bug.cgi?id=578382

This means we have to wait for 2022/03 to release. Which sounds like a solid target to me.

[hvbtup]

Thanks for your effort, Wim.

Meanwhile I tried to find out how to cope with the OSGI runtime.

I did not find any way to direct the .manager directories etc to a different location than ReportEngine/platform/configuration.
So I set that whole directory read-only, and I can still generate reports.
I don't know, however, if this restricts functionality somehow or if the runtime will create those files somewhere else.
IMHO the OSGI runtime is still not a reasonable solution if BIRT shall run in several parallel processes (as a service on an application server).

So I'll wait until 2022/03 and hope that the POJO runtime will work then.

Will this prevent us from a new BIRT release then?

Or should we release nevertheless and only mention that the POJO runtime isn't working at the moment, but will come with the next release then?

[patric-r]

Thanks!

[wimjongman]

@hvbtup said: Meanwhile I tried to find out how to cope with the OSGI runtime.

Maybe you can convert that question into a discussion? You have to be careful not to pollute issues with side questions too much. It makes it very hard to keep an overview.

Please start with a discussion first so that we can then make separate issues. I think you do not pass the OSGi instance area information correctly but this question has no business in this issue.

[wimjongman]

This issue has been partially resolved here. The final fix has been made in the eclipse platform.

[wimjongman]

Henning, can you retry? The latest build contains the platform patch.

[hvbtup]

@wimjongman You are my hero for today!
I can confirm that is does work now with the POJO runtime.