New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NPE in GlassFishORBHelper.getORB #1967

Closed
dmatej opened this Issue Sep 12, 2017 · 11 comments

Comments

Projects
None yet
4 participants
@dmatej
Contributor

dmatej commented Sep 12, 2017

Description


NullpointerException in GlassFishORBHelper:163 when trying to call ping-connection-pool in prepare-resources.txt.

It seems the orbFactory property is not initialized (Inject annotation is commented out, postConstruct method is not called yet, maybe it has some reason).

Expected Outcome

Command execution causing successful verification of the JDBC pool configuration.
OR
Exception with a comprehensible message, f.e. "ORB not initialized yet." - to give some hint what to do (to move the command to be executed later after the ORB initialization).

Current Outcome

java.lang.RuntimeException: Orb initialization erorr
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:191)
        at com.sun.enterprise.naming.impl.SerialContext.getORB(SerialContext.java:349)
        at com.sun.enterprise.naming.impl.SerialContext.getProviderCacheKey(SerialContext.java:356)
        at com.sun.enterprise.naming.impl.SerialContext.getRemoteProvider(SerialContext.java:386)
        at com.sun.enterprise.naming.impl.SerialContext.getProvider(SerialContext.java:331)
        at com.sun.enterprise.naming.impl.SerialContext.lookup(SerialContext.java:480)
        at com.sun.enterprise.naming.impl.SerialContext.lookup(SerialContext.java:440)
        at javax.naming.InitialContext.lookup(InitialContext.java:417)
        at org.glassfish.resourcebase.resources.naming.ResourceNamingService.lookup(ResourceNamingService.java:236)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.getConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:799)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.obtainManagedConnectionFactory(ConnectorConnectionPoolAdminServiceImpl.java:938)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.getUnpooledConnection(ConnectorConnectionPoolAdminServiceImpl.java:549)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.testConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:430)
        at com.sun.enterprise.connectors.ConnectorRuntime.pingConnectionPool(ConnectorRuntime.java:1162)
        at org.glassfish.connectors.admin.cli.PingConnectionPool.execute(PingConnectionPool.java:143)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:544)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:540)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:539)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:570)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:562)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:561)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1469)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:111)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1851)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1727)
        at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.executeCommand(CommandExecutorImpl.java:169)
        at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.run(CommandExecutorImpl.java:94)
        at fish.payara.micro.boot.runtime.BootCommand.execute(BootCommand.java:65)
        at fish.payara.micro.boot.runtime.BootCommands.executeCommands(BootCommands.java:105)
        at fish.payara.micro.boot.runtime.BootCommands.executeCommands(BootCommands.java:99)
        at fish.payara.micro.impl.PayaraMicroImpl.bootStrap(PayaraMicroImpl.java:987)
        at fish.payara.micro.impl.PayaraMicroImpl.main(PayaraMicroImpl.java:186)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at fish.payara.micro.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:107)
        at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:70)
        at fish.payara.micro.boot.PayaraMicroLauncher.main(PayaraMicroLauncher.java:79)
        at fish.payara.micro.PayaraMicro.main(PayaraMicro.java:361)
Caused by: java.lang.NullPointerException
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:163)
        ... 44 more
java.lang.RuntimeException: Orb initialization erorr
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:191)
        at com.sun.enterprise.naming.impl.SerialContext.getORB(SerialContext.java:349)
        at com.sun.enterprise.naming.impl.SerialContext.getProviderCacheKey(SerialContext.java:356)
        at com.sun.enterprise.naming.impl.SerialContext.getRemoteProvider(SerialContext.java:386)
        at com.sun.enterprise.naming.impl.SerialContext.getProvider(SerialContext.java:331)
        at com.sun.enterprise.naming.impl.SerialContext.rebind(SerialContext.java:678)
        at com.sun.enterprise.naming.impl.SerialContext.rebind(SerialContext.java:695)
        at javax.naming.InitialContext.rebind(InitialContext.java:437)
        at com.sun.enterprise.naming.impl.GlassfishNamingManagerImpl.publishObject(GlassfishNamingManagerImpl.java:210)
        at com.sun.enterprise.naming.impl.GlassfishNamingManagerImpl.publishObject(GlassfishNamingManagerImpl.java:196)
        at org.glassfish.resourcebase.resources.naming.ResourceNamingService.publishObject(ResourceNamingService.java:168)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.createConnectorConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:190)
        at com.sun.enterprise.connectors.ConnectorRuntime.createConnectorConnectionPool(ConnectorRuntime.java:331)
        at org.glassfish.jdbc.deployer.JdbcConnectionPoolDeployer.actualDeployResource(JdbcConnectionPoolDeployer.java:201)
        at org.glassfish.jdbc.deployer.JdbcConnectionPoolDeployer.deployResource(JdbcConnectionPoolDeployer.java:170)
        at com.sun.enterprise.connectors.service.ConnectorService.loadDeferredResources(ConnectorService.java:233)
        at com.sun.enterprise.connectors.service.ConnectorService$1.run(ConnectorService.java:153)
        at java.security.AccessController.doPrivileged(Native Method)
        at com.sun.enterprise.connectors.service.ConnectorService.loadResourcesAndItsRar(ConnectorService.java:150)
        at com.sun.enterprise.connectors.service.ConnectorService.checkAndLoadPool(ConnectorService.java:325)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.getUnpooledConnection(ConnectorConnectionPoolAdminServiceImpl.java:553)
        at com.sun.enterprise.connectors.service.ConnectorConnectionPoolAdminServiceImpl.testConnectionPool(ConnectorConnectionPoolAdminServiceImpl.java:430)
        at com.sun.enterprise.connectors.ConnectorRuntime.pingConnectionPool(ConnectorRuntime.java:1162)
        at org.glassfish.connectors.admin.cli.PingConnectionPool.execute(PingConnectionPool.java:143)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:544)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2$1.run(CommandRunnerImpl.java:540)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$2.execute(CommandRunnerImpl.java:539)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:570)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$3.run(CommandRunnerImpl.java:562)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:360)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:561)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.doCommand(CommandRunnerImpl.java:1469)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl.access$1300(CommandRunnerImpl.java:111)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1851)
        at com.sun.enterprise.v3.admin.CommandRunnerImpl$ExecutionContext.execute(CommandRunnerImpl.java:1727)
        at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.executeCommand(CommandExecutorImpl.java:169)
        at com.sun.enterprise.admin.cli.embeddable.CommandExecutorImpl.run(CommandExecutorImpl.java:94)
        at fish.payara.micro.boot.runtime.BootCommand.execute(BootCommand.java:65)
        at fish.payara.micro.boot.runtime.BootCommands.executeCommands(BootCommands.java:105)
        at fish.payara.micro.boot.runtime.BootCommands.executeCommands(BootCommands.java:99)
        at fish.payara.micro.impl.PayaraMicroImpl.bootStrap(PayaraMicroImpl.java:987)
        at fish.payara.micro.impl.PayaraMicroImpl.main(PayaraMicroImpl.java:186)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at fish.payara.micro.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:107)
        at fish.payara.micro.boot.loader.Launcher.launch(Launcher.java:70)
        at fish.payara.micro.boot.PayaraMicroLauncher.main(PayaraMicroLauncher.java:79)
        at fish.payara.micro.PayaraMicro.main(PayaraMicro.java:361)
Caused by: java.lang.NullPointerException
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:163)
        ... 53 more
[2017-09-12T14:08:37.069+0200] [] [SEVERE] [] [javax.enterprise.resource.resourceadapter.com.sun.enterprise.connectors.service] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1505218117069] [levelValue: 1000] RAR6020 : Failed to publish connection pool object in JNDI : metering-mysql

Steps to reproduce (Only for bug reports)

   <plugins>
      <!-- HOWTO: https://github.com/payara/maven-plugins -->
      <plugin>
        <groupId>fish.payara.maven.plugins</groupId>
        <artifactId>payara-micro-maven-plugin</artifactId>
        <version>1.0.0</version>
        <executions>
          <execution>
            <goals>
              <goal>bundle</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <payaraVersion>4.1.2.173</payaraVersion>
          <useUberJar>true</useUberJar>
          <deployArtifacts>
            <deployArtifact>
              <groupId>cz.netpro.metering2</groupId>
              <artifactId>metering2-ws</artifactId>
              <version>${project.version}</version>
              <type>war</type>
            </deployArtifact>
          </deployArtifacts>
          <customJars>
            <customJar>
              <groupId>mysql</groupId>
              <artifactId>mysql-connector-java</artifactId>
            </customJar>
          </customJars>
          <commandLineOptions>
            <commandLineOption>
              <key>--autobindhttp</key>
              <value>true</value>
            </commandLineOption>
            <commandLineOption>
              <key>--prebootcommandfile</key>
              <value>${project.build.outputDirectory}/prepare-resources.txt</value>
            </commandLineOption>
          </commandLineOptions>
        </configuration>
      </plugin>
    </plugins>

set configs.config.server-config.admin-service.das-config.dynamic-reload-enabled=false
set configs.config.server-config.admin-service.das-config.autodeploy-enabled=false
create-jdbc-connection-pool --datasourceclassname com.mysql.cj.jdbc.MysqlDataSource --restype javax.sql.DataSource --property user=xxx:password=xxx:DatabaseName=xxxxxxxx:ServerName=localhost:port=3306:zeroDateTimeBehavior=convertToNull:useUnicode=true:characterEncoding=UTF-8 xxx-mysql
set resources.jdbc-connection-pool.metering-mysql.steady-pool-size=5
set resources.jdbc-connection-pool.metering-mysql.max-pool-size=20
set resources.jdbc-connection-pool.metering-mysql.connection-validation-method=auto-commit
set resources.jdbc-connection-pool.metering-mysql.is-connection-validation-required=true
set resources.jdbc-connection-pool.metering-mysql.fail-all-connections=true
ping-connection-pool metering-mysql

The last command fails. There are more stakctraces hinting that my prebootcommandfile is not correct, but none of them says that directly. NPE is really ugly.

Samples

No sample

Context (Optional)

First attempts to use Payara Micro with JAX-RS, JPA, JSF and JTA.
Workaround (or perhaps undocumented solution): move ping-connection-pool to the postbootcommandfile

Environment

  • Payara Version: 4.1.2.173
  • Edition: Micro
  • JDK Version: Oracle 8u144
  • Operating System: Linux Kubuntu 17.04
  • Database: MySQL
@fturizo

This comment has been minimized.

Contributor

fturizo commented Sep 22, 2017

Greetings @dmatej

In this case, Payara Micro is behaving accordingly based on the configuration you have set up. According to the official docs, there are some restrictions on asadmin commands that are run as part of a preboot-command script. In this case, the ping-connection-pools command seems to be in need to initialize internal components which require that the ORB is ready (which is not since the server hasn't properly booted), which causes the error to be thrown. The workaround you have proposed works since at post-boot time, the server is ready to run the command.

Hope this clears the issue.

@lprimak

This comment has been minimized.

Contributor

lprimak commented Sep 22, 2017

We may still want to fix the NPE though, it's a simple fix.
Feel free to assign the issue to me, it's very minor

@dmatej

This comment has been minimized.

Contributor

dmatej commented Sep 22, 2017

I already discussed it with Ondrej, if you agree with only adding a message I can fix it today, I should have some time for that ... or you already did it? :-)

@fturizo NPE without a message is always a bug.

@lprimak

This comment has been minimized.

Contributor

lprimak commented Sep 22, 2017

No, I didn't do it, and a fix is always welcome, thanks!

@fturizo

This comment has been minimized.

Contributor

fturizo commented Sep 23, 2017

I agree @dmatej! Thanks for working on a possible fix then!

@dmatej

This comment has been minimized.

Contributor

dmatej commented Sep 26, 2017

I got some bad news and some good news ... since I rebooted Payara Micro never started from Maven plugin with any version of Payara Micro (current snapshot 4.1.2.174 I have built, 4.1.2.173, 4.1.2.172), it crashed with always the same NPE on the same line. I removed all the pre and post scripts but nothing helped. It took me four hours until I remembered that I tried to install and use OpenJDK9 from Kubuntu repository. It did matter despite I used JDK8 in .mavenrc and JAVA_HOME.
Same problem had Payara build, it could not even generate any domain.xml - the problem is ServiceLocator, it does not work, but I don't know what apt installer changed in system causing this.
After uninstall of the JDK9 package Payara Micro started with no other change and Payara builds were ok too.
One note: If generating domain.xml in Maven build of Payara fails, Maven build is still successful. I think it should not. But that is another issue.

It seems that the ORB initialization is completely evily wrong - thread race conditions and "dummy workarounds" in the code - see synchronized blocks, methods, double-checked locking, commented out @Inject annotation, mixing factory/service initialization and setter/getter, getters using the factory that might not have been initialized, TODOs in other classes ... this code makes me mad ... but I have no time to fix it all and even worse, I cannot understand all mechanisms (hacks) working with this.

Today I will push some small changes in GlassFishORBHelper (NPE with a message), but I think there is much more work needed on the mechanism. But it should work for now ...

@lprimak

This comment has been minimized.

Contributor

lprimak commented Sep 26, 2017

We are going to upgrade base CORBA package soon as well

@lprimak

This comment has been minimized.

Contributor

lprimak commented Sep 26, 2017

i.s. the CORBA on which that Payara patched CORBA is based on.
Doubt that they have reworked the initialization code though :)

@dmatej

This comment has been minimized.

Contributor

dmatej commented Sep 26, 2017

Yep. I had issues with building current snapshot of Payara4 - embedded is built only with a profile but crashes due some weird Ant error ... (some dependency packaged with incompatible ZIP algorithm?), so I patched 4.1.2.173.0.1 version - and tested both with embedded and micro distributions. Both work but for micro it seems ORB factory is not initialized at all and it uses setOrb method and does no calls to getters (because I have noticed no errors); so the message level I will lower before commit ...:

[2017-09-26T22:49:45.118+0200] [] [WARNING] [] [javax.enterprise.resource.corba.org.glassfish.enterprise.iiop.api] [tid: _ThreadID=1 _ThreadName=main] [timeMillis: 1506458985118] [levelValue: 900] GlassFishORBFactory service not found.

Embedded distribution initializes via the ServiceLocator ...

Zář 26, 2017 10:49:24 ODP. org.glassfish.enterprise.iiop.api.GlassFishORBHelper postConstruct
INFO: GlassFishORBFactory service initialized.

And the stacktrace when I add the "ping" to will now look like this (not much difference, but ...):

Caused by: java.lang.NullPointerException: GlassFishORBFactory is not initialized!
        at java.util.Objects.requireNonNull(Objects.java:228)
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORBFactory(GlassFishORBHelper.java:243)
        at org.glassfish.enterprise.iiop.api.GlassFishORBHelper.getORB(GlassFishORBHelper.java:167)
        ... 44 more
@chrisxaustin

This comment has been minimized.

chrisxaustin commented Oct 22, 2018

I was seeing this error while migrating an app from normal payara to payara-micro.
The old app had @Remote interfaces still, and once I removed them it started up normally.

Adding a comment to this issue in case anyone else gets stuck on the same problem.

@lprimak

This comment has been minimized.

Contributor

lprimak commented Oct 22, 2018

Payara Micro does not support remote EJBs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment