fileMetadataExtractableFromNetcdf causing regression for S3 direct upload #9601

pdurbin · 2023-05-19T15:40:29Z

The fileMetadataExtractableFromNetcdf method I added in this pull request...

extract bounding box from NetCDF/HDF5 files and insert into the geospatial metadata block #9541

... is causing a regression when S3 direct upload is enabled. Here's how the UI looks.

Stacktrace

[#|2023-05-19T15:33:45.613+0000|WARNING|Payara 5.2022.4|javax.enterprise.ejb.container|_ThreadID=99;_ThreadName=http-thread-pool::http-listener-1(5);_TimeMillis=1684510425613;_LevelValue=900;_MessageID=AS-EJB-00056;|
A system exception occurred during an invocation on EJB IngestServiceBean, method: public java.util.List edu.harvard.iq.dataverse.ingest.IngestServiceBean.saveAndAddFilesToDataset(edu.harvard.iq.dataverse.DatasetVersion,java.util.List,edu.harvard.iq.dataverse.DataFile,boolean)|#]

[#|2023-05-19T15:33:45.614+0000|WARNING|Payara 5.2022.4|javax.enterprise.ejb.container|_ThreadID=99;_ThreadName=http-thread-pool::http-listener-1(5);_TimeMillis=1684510425614;_LevelValue=900;|
javax.ejb.EJBException
at com.sun.ejb.containers.EJBContainerTransactionManager.processSystemException(EJBContainerTransactionManager.java:723)
at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:652)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:482)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4601)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2134)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2104)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
at com.sun.proxy.$Proxy377.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.ingest.EJB31_Generated__IngestServiceBean__Intf____Bean.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.EditDatafilesPage.save(EditDatafilesPage.java:1067)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
at javax.faces.component.UICommand.broadcast(UICommand.java:222)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1637)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:167)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:372)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:239)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:201)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.NullPointerException
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:828)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:818)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.open(S3AccessIO.java:210)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.fileMetadataExtractableFromNetcdf(IngestServiceBean.java:1200)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.saveAndAddFilesToDataset(IngestServiceBean.java:372)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at jdk.internal.reflect.GeneratedMethodAccessor220.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at jdk.internal.reflect.GeneratedMethodAccessor140.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
... 61 more
|#]

[#|2023-05-19T15:33:45.618+0000|WARNING|Payara 5.2022.4|javax.enterprise.resource.webcontainer.jsf.lifecycle|_ThreadID=99;_ThreadName=http-thread-pool::http-listener-1(5);_TimeMillis=1684510425618;_LevelValue=900;|
#{EditDatafilesPage.save}: javax.ejb.EJBException
javax.faces.FacesException: #{EditDatafilesPage.save}: javax.ejb.EJBException
at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:96)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
at javax.faces.component.UICommand.broadcast(UICommand.java:222)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1637)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:167)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:372)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:239)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:201)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.faces.el.EvaluationException: javax.ejb.EJBException
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:76)
at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
... 45 more
Caused by: javax.ejb.EJBException
at com.sun.ejb.containers.EJBContainerTransactionManager.processSystemException(EJBContainerTransactionManager.java:723)
at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:652)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:482)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4601)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2134)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2104)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
at com.sun.proxy.$Proxy377.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.ingest.EJB31_Generated__IngestServiceBean__Intf____Bean.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.EditDatafilesPage.save(EditDatafilesPage.java:1067)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
... 46 more
Caused by: java.lang.NullPointerException
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:828)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:818)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.open(S3AccessIO.java:210)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.fileMetadataExtractableFromNetcdf(IngestServiceBean.java:1200)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.saveAndAddFilesToDataset(IngestServiceBean.java:372)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at jdk.internal.reflect.GeneratedMethodAccessor220.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at jdk.internal.reflect.GeneratedMethodAccessor140.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
... 61 more
|#]

[#|2023-05-19T15:33:45.623+0000|SEVERE|Payara 5.2022.4|javax.enterprise.resource.webcontainer.jsf.context|_ThreadID=99;_ThreadName=http-thread-pool::http-listener-1(5);_TimeMillis=1684510425623;_LevelValue=1000;|
javax.faces.el.EvaluationException: javax.ejb.EJBException
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:76)
at com.sun.faces.application.ActionListenerImpl.getNavigationOutcome(ActionListenerImpl.java:82)
at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:71)
at javax.faces.component.UICommand.broadcast(UICommand.java:222)
at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:847)
at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1396)
at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:58)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:76)
at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:177)
at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:707)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1637)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:282)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:226)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:253)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:257)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:167)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:757)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:577)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:158)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:372)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:239)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:520)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:217)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:182)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:156)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:201)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:95)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:260)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:177)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:109)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:88)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:53)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:524)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:89)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:94)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:33)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:114)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549)
at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.ejb.EJBException
at com.sun.ejb.containers.EJBContainerTransactionManager.processSystemException(EJBContainerTransactionManager.java:723)
at com.sun.ejb.containers.EJBContainerTransactionManager.completeNewTx(EJBContainerTransactionManager.java:652)
at com.sun.ejb.containers.EJBContainerTransactionManager.postInvokeTx(EJBContainerTransactionManager.java:482)
at com.sun.ejb.containers.BaseContainer.postInvokeTx(BaseContainer.java:4601)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2134)
at com.sun.ejb.containers.BaseContainer.postInvoke(BaseContainer.java:2104)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:220)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:90)
at com.sun.proxy.$Proxy377.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.ingest.EJB31_Generated__IngestServiceBean__Intf____Bean.saveAndAddFilesToDataset(Unknown Source)
at edu.harvard.iq.dataverse.EditDatafilesPage.save(EditDatafilesPage.java:1067)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:163)
at com.sun.el.parser.AstValue.invoke(AstValue.java:261)
at com.sun.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:237)
at org.jboss.weld.module.web.util.el.ForwardingMethodExpression.invoke(ForwardingMethodExpression.java:40)
at org.jboss.weld.module.web.el.WeldMethodExpression.invoke(WeldMethodExpression.java:50)
at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:65)
at com.sun.faces.application.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:66)
... 46 more
Caused by: java.lang.NullPointerException
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:828)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.getMainFileKey(S3AccessIO.java:818)
at edu.harvard.iq.dataverse.dataaccess.S3AccessIO.open(S3AccessIO.java:210)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.fileMetadataExtractableFromNetcdf(IngestServiceBean.java:1200)
at edu.harvard.iq.dataverse.ingest.IngestServiceBean.saveAndAddFilesToDataset(IngestServiceBean.java:372)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:588)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:408)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:4835)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:665)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:834)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doCall(SystemInterceptorProxy.java:163)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:140)
at jdk.internal.reflect.GeneratedMethodAccessor220.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:615)
at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72)
at org.jboss.weld.module.ejb.SessionBeanInterceptor.aroundInvoke(SessionBeanInterceptor.java:52)
at jdk.internal.reflect.GeneratedMethodAccessor140.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.base/java.lang.reflect.Method.invoke(Unknown Source)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:888)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:833)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:375)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:4807)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:4795)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:212)
... 61 more
|#]

Tested with a NetCDF file and S3 direct upload enabled.

cmbz · 2023-05-22T19:40:51Z

Sized as 33 after conversation with Phil

Note that the NcML aux file is not created when S3 direct upload is enabled.

By default, keep S3 direct upload fast. Don't download NetCDF or HDF5 files to try to pull geospatial metadata out of them when S3 direct upload is configured. If you really want this, add this setting and make it true.

Conflicts: doc/sphinx-guides/source/installation/config.rst src/main/java/edu/harvard/iq/dataverse/ingest/IngestServiceBean.java

pdurbin added this to the 5.14 milestone May 19, 2023

pdurbin self-assigned this May 19, 2023

pdurbin added this to ▶ SPRINT READY in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) via automation May 19, 2023

pdurbin mentioned this issue May 19, 2023

extract bounding box from NetCDF/HDF5 files and insert into the geospatial metadata block #9541

Merged

pdurbin added a commit that referenced this issue May 19, 2023

catch ex to prevent NPE on save in GUI #9601

550483f

Tested with a NetCDF file and S3 direct upload enabled.

qqmyers added a commit to GlobalDataverseCommunityConsortium/dataverse that referenced this issue May 19, 2023

fix for IQSS#9601

57e984b

pdurbin moved this from ▶ SPRINT READY to IQSS Team - In Progress 💻 in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) May 22, 2023

cmbz added the Size: 30 A percentage of a sprint. 21 hours. (formerly size:33) label May 22, 2023

pdurbin added a commit that referenced this issue May 23, 2023

disable NetCDF metadata extraction for S3 direct upload #9601

6838fdf

pdurbin added a commit that referenced this issue May 23, 2023

set expectations about S3 direct upload behavior #9601

046a049

pdurbin mentioned this issue May 23, 2023

disable NetCDF/HDF5 metadata extraction for S3 direct upload #9609

Closed

pdurbin added a commit that referenced this issue May 23, 2023

fix S3 direct upload NPE and keep NetCDF metadata extraction #9601

cf2088d

Note that the NcML aux file is not created when S3 direct upload is enabled.

pdurbin removed this from IQSS Team - In Progress 💻 in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) May 23, 2023

pdurbin removed their assignment May 23, 2023

pdurbin mentioned this issue May 23, 2023

fix S3 direct upload NPE and NetCDF/HDF5 and keep geospatial metadata extraction #9611

Merged

pdurbin added a commit that referenced this issue May 24, 2023

set expectations about S3 direct upload behavior #9601

8f83738

pdurbin added this to ▶ SPRINT READY in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) via automation May 24, 2023

pdurbin moved this from ▶ SPRINT READY to IQSS Team - In Progress 💻 in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) May 24, 2023

pdurbin self-assigned this May 24, 2023

pdurbin added a commit that referenced this issue May 24, 2023

switch to mime type check #9601

505e8f2

pdurbin removed this from IQSS Team - In Progress 💻 in IQSS/dataverse (TO BE RETIRED / DELETED in favor of project 34) May 25, 2023

pdurbin removed their assignment May 25, 2023

pdurbin added a commit that referenced this issue May 25, 2023

turn down logging #9601

5a0bf24

pdurbin added a commit that referenced this issue May 26, 2023

set expectations about NcML files (modern only) #9153 #9601

2d6564a

qqmyers added a commit to GlobalDataverseCommunityConsortium/dataverse that referenced this issue May 30, 2023

fix for IQSS#9601

bdce3b4

pdurbin added a commit that referenced this issue Jun 15, 2023

Merge branch 'develop' into 9601-keep-features #9601

1b6f4b2

Conflicts: doc/sphinx-guides/source/installation/config.rst src/main/java/edu/harvard/iq/dataverse/ingest/IngestServiceBean.java

pdurbin added a commit that referenced this issue Jul 10, 2023

Merge branch 'develop' into 9601-keep-features #9601

bbea224

kcondon closed this as completed in #9611 Jul 10, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fileMetadataExtractableFromNetcdf causing regression for S3 direct upload #9601

fileMetadataExtractableFromNetcdf causing regression for S3 direct upload #9601

pdurbin commented May 19, 2023

cmbz commented May 22, 2023

fileMetadataExtractableFromNetcdf causing regression for S3 direct upload #9601

fileMetadataExtractableFromNetcdf causing regression for S3 direct upload #9601

Comments

pdurbin commented May 19, 2023

cmbz commented May 22, 2023