JobOperatorImpl cannot be instantiated

[ivangonzalezsaenz]

Hello, I'm having an issue when trying to run a test on JBatch. I'm receiving the following errors on server.log (debug mode):

Grave: Could not instantiate service: fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager due to exception:java.lang.ClassNotFoundException: fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager not found by org.glassfish.main.batch.payara-jbatch [241]
Grave: Could not instantiate service: com.ibm.jbatch.container.impl.BatchKernelImpl due to exception:java.lang.reflect.InvocationTargetException
Advertencia: Exception while dispatching an event
java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider com.ibm.jbatch.container.api.impl.JobOperatorImpl could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.batch.runtime.BatchRuntime$1.run(BatchRuntime.java:51)
at javax.batch.runtime.BatchRuntime$1.run(BatchRuntime.java:46)
at java.security.AccessController.doPrivileged(Native Method)
at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:46)
at org.glassfish.batch.spi.impl.BatchRuntimeHelper.event(BatchRuntimeHelper.java:221)
at org.glassfish.kernel.event.EventsImpl$1.run(EventsImpl.java:123)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Could not instantiate service com.ibm.jbatch.container.impl.BatchKernelImpl
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadServiceHelper(ServicesManagerImpl.java:424)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.getService(ServicesManagerImpl.java:403)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.access$300(ServicesManagerImpl.java:388)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl.getService(ServicesManagerImpl.java:304)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl.getBatchKernelService(ServicesManagerImpl.java:348)
at com.ibm.jbatch.container.api.impl.JobOperatorImpl.(JobOperatorImpl.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
... 13 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadService(ServicesManagerImpl.java:447)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadServiceHelper(ServicesManagerImpl.java:420)
... 24 more
Caused by: java.lang.RuntimeException: Could not instantiate service fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadServiceHelper(ServicesManagerImpl.java:424)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.getService(ServicesManagerImpl.java:403)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader.access$300(ServicesManagerImpl.java:388)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl.getService(ServicesManagerImpl.java:304)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl.getPersistenceManagerService(ServicesManagerImpl.java:333)
at com.ibm.jbatch.container.impl.BatchKernelImpl.(BatchKernelImpl.java:87)
... 30 more
Caused by: java.lang.ClassNotFoundException: fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager not found by org.glassfish.main.batch.payara-jbatch [241]
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1532)
at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:75)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:1955)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadService(ServicesManagerImpl.java:441)
at com.ibm.jbatch.container.servicesmanager.ServicesManagerImpl$ServiceLoader._loadServiceHelper(ServicesManagerImpl.java:420)
... 35 more

Advertencia: RAR8068: Using default datasource : __ds_jdbc_ra for pool : java:app/fact_electPool
Información: EclipseLink, version: Eclipse Persistence Services - 2.6.2.qualifier
Información: /file:/C:/Users/ivan.gonzalez.saenz/workspace/FactElectronica/FactElectronica-ear/target/gfdeploy/FactElectronica-ear/FactElectronica-web-1.0-SNAPSHOT_war/WEB-INF/lib/FactElectronica-ejb-1.0-SNAPSHOT.jar_FactElectronicaPU login successful
Información: Portable JNDI names for EJB NewSessionBean: [java:global/FactElectronica-ear/FactElectronica-ejb-1.0-SNAPSHOT/NewSessionBean!com.igonzalez.NewSessionBean, java:global/FactElectronica-ear/FactElectronica-ejb-1.0-SNAPSHOT/NewSessionBean]
Información: Portable JNDI names for EJB NewSessionBean: [java:global/FactElectronica-ear/FactElectronica-web-1.0-SNAPSHOT/NewSessionBean, java:global/FactElectronica-ear/FactElectronica-web-1.0-SNAPSHOT/NewSessionBean!com.igonzalez.NewSessionBean]
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: WELD-000411: Observer method [BackedAnnotatedMethod] private org.glassfish.jersey.ext.cdi1x.internal.CdiComponentProvider.processAnnotatedType(@observes ProcessAnnotatedType) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
Información: Inicializando Mojarra 2.2.13 ( 20160203-1910 unable to get svn info) para el contexto '/FactElectronica-web'
Información: Loading application [FactElectronica-ear#FactElectronica-web-1.0-SNAPSHOT.war] at [/FactElectronica-web]
Información: FactElectronica-ear was successfully deployed in 2.141 milliseconds.
Advertencia: Context path from ServletContext: differs from path from bundle: /
Información: Redirecting to /common/index.jsf
Información: Admin Console: Initializing Session Attributes...
Advertencia: Could not open/create prefs root node Software\JavaSoft\Prefs at root 0x80000002. Windows RegCreateKeyEx(...) returned error code 5.

[mikecroft]

We will need a lot more information to be able to help on this.

• What was the test?
• What was the expected outcome?
• What was the actual outcome?
• Can you upload, or link to a repository of a Short, Self Contained, Correct (Compilable), Example?
• What version of Payara Server are you using?

[ivangonzalezsaenz]

Hi Mike,
The test is for using an ItemReader implementation using chunk and steps to read from a CSV file and write to a database, using a Java batch persistence configuration with a MySQL database for the job repository. I kept trying to solve it yesterday and today and after adding the following dependencies in the web module pom file, the application finally found the fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager :

        <dependency>
            <groupId>com.ibm.jbatch</groupId>
            <artifactId>com.ibm.jbatch.spi</artifactId>
            <version>1.0.1</version>
        </dependency>
        <dependency>
            <groupId>com.ibm.jbatch</groupId>
            <artifactId>com.ibm.jbatch.container</artifactId>
            <version>1.0.1</version>
        </dependency>

, However, i still receive a NPE for the JobOperatorImpl object after I executed:
jo = BatchRuntime.getJobOperator();
long jid = jo.start("fact_elect_job", null);

This is now the error log:

Grave:   **javax.batch.operations.JobStartException: java.lang.NullPointerException
    at com.ibm.jbatch.container.api.impl.JobOperatorImpl.start(JobOperatorImpl.java:92)**
    at com.igonzalez.batch.FactElectBatchServlet.processRequest(FactElectBatchServlet.java:53)
    at com.igonzalez.batch.FactElectBatchServlet.doGet(FactElectBatchServlet.java:67)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
    at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1681)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:318)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
    at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
    at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
    at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
    at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:416)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:283)
    at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:466)
    at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:169)
    at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:206)
    at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:180)
    at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
    at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
    at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
    at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
    at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:526)
    at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
    at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:591)
    at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:571)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
    at fish.payara.jbatch.persistence.rdbms.LazyBootPersistenceManager.createJobInstance(LazyBootPersistenceManager.java:170)
    at com.ibm.jbatch.container.jobinstance.JobExecutionHelper.getNewJobInstance(JobExecutionHelper.java:89)
    at com.ibm.jbatch.container.jobinstance.JobExecutionHelper.startJob(JobExecutionHelper.java:120)
    at com.ibm.jbatch.container.impl.BatchKernelImpl.startJob(BatchKernelImpl.java:124)
    at com.ibm.jbatch.container.api.impl.JobOperatorImpl.startInternal(JobOperatorImpl.java:123)
    at com.ibm.jbatch.container.api.impl.JobOperatorImpl.start(JobOperatorImpl.java:88)
    ... 32 more

The actual outcome is that the application is not recognizing the current batch runtime persistence configuration that I made on Payara for the Job Repository (MySQL) and instead is trying to use the default Derby database configuration (see file attached).

The expected outcome is to accomplish a connection to my MySQL database configured as the job Repository

The version of Payara is "Payara Server 163"

Thanks.

[smillidge]

Can you make sure there are no JBatch jars in your ear file. All the dependencies should be provided scope

[ivangonzalezsaenz]

Steve, I removed all dependencies and the issue is still the same.

[smillidge]

Can you try this sample application?

https://github.com/payara/Payara-Examples/tree/master/Java-EE/jbatch-schedule

[smillidge]

I tested against MySQL with the example application and had no problem. Here's an asadmin script to configure MySQL generated using the asadmin recorder in the admin console. The script assumes your mysql schema name, the user and the password are all payara.
You also need to copy the MySQL JDBC Driver jar into the glassfish/lib directory.

create-jdbc-connection-pool --validateAtmostOncePeriodInSeconds=0 --statementLeakReclaim=false --ping=false --maxWaitTimeInMillis=60000 --wrapJdbcObjects=true --connectionValidationMethod=table --steadyPoolSize=8 --idleTimeoutInSeconds=300 --maxConnectionUsageCount=0 --matchConnections=false --connectionCreationRetryIntervalInSeconds=10 --associateWithThread=false --isConnectionValidationRequired=false --connectionLeakReclaim=false --datasourceClassname=com.mysql.jdbc.jdbc2.optional.MysqlDataSource --connectionCreationRetryAttempts=0 --statementCacheSize=0 --resType=javax.sql.DataSource --statementLeakTimeoutInSeconds=0 --failAllConnections=false --statementTimeoutInSeconds=-1 --lazyConnectionEnlistment=false --pooling=true --poolResizeQuantity=2 --allowNonComponentCallers=false --connectionLeakTimeoutInSeconds=0 --isIsolationLevelGuaranteed=true --maxPoolSize=32 --nonTransactionalConnections=false --lazyConnectionAssociation=false NewBatchPool

set resources.jdbc-connection-pool.NewBatchPool.property.User=payara resources.jdbc-connection-pool.NewBatchPool.property.Password=payara resources.jdbc-connection-pool.NewBatchPool.property.ZeroDateTimeBehavior=convertToNull

create-jdbc-resource --enabled=true --poolName=NewBatchPool --target=domain jdbc/BatchPool

create-resource-ref --enabled=true --target=server jdbc/BatchPool

set-batch-runtime-configuration --tableSuffix= --tablePrefix= --dataSourceLookupName=jdbc/BatchPool --executorServiceLookupName=concurrent/__defaultManagedExecutorService --schemaName=payara --target=server-config 

[ivangonzalezsaenz]

The issue keeps the same, even now that I'm using the new BatchPool resource. I think there is something with the variables stored in batchContainerProps corresponding to the ServicesManagerImpl object of my JobOperatorImpl. Values are still pointing to a Derby database. Thanks.

[smillidge]

There are only three code paths where that NPE can occur that I can see.

1. The Datasource name you have configured for the Batch subsystem is not a datasource
2. The database is returning a product name string that is not recognised.
3. The datasource can not connect to the database to get a connection (in which case a SEVERE log message should precede the error.

The only other option is that you have jars from JBatch in your WEB-INF/lib in your war file. I can't see how else the NPE could be thrown.

[ivangonzalezsaenz]

Steve, I finally could solve it. I activated debug mode on both the server and on Netbeans. First, I noticed that I received the following message:

Unknown database 'payara'

, and I found that Glassfish was trying to interpret that the schema that we configured days ago as 'payara' was misunderstood as if it were the "database". Then I remember an article I read here, (see section "3.4 Specifying a Blank Schema Name") so I left empty the schema property, however I received this other message in the log:

**Grave:   Unknown database 'test'**
Grave:   Could not instantiate service: com.ibm.jbatch.container.impl.BatchKernelImpl due to exception:java.lang.reflect.InvocationTargetException
Listening for transport dt_socket at address: 9009
Advertencia:   StandardWrapperValve[FactElectBatchServlet]: Servlet.service() for servlet FactElectBatchServlet threw exception
java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider com.ibm.jbatch.container.api.impl.JobOperatorImpl could not be instantiated
    at java.util.ServiceLoader.fail(ServiceLoader.java:232)
    at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at javax.batch.runtime.BatchRuntime$1.run(BatchRuntime.java:51)
    at javax.batch.runtime.BatchRuntime$1.run(BatchRuntime.java:46)

Finally, I renamed the JDBC resource, from jdbc/batchpool to jdbc/test (as it is also explained on the previous document) and it worked!!

Thanks!

[smillidge]

Great we need to make these errors more obvious by suppressing the stack traces to make it easy to spot problems like this.