Problems with execution info in Neo4J #8204
Replies: 2 comments 2 replies
|
do you get any errors or other information in the console you used to start Hop Gui? |
2 replies
|
Sorry for not answering before. Yes, after 5min or more I get the following error (details). It's like something related to the size of the commit, but I'm not sure how to split the execution information in more commits or at least select what information or content of the fields to store. Because for example, I'm retrieving JSON information from an API I'm working with, and I'm only interested in the extracted fields, not the JSON body that the API returns (because it's huge and I deduce it's the reason why the commit is so big). Details
2026/09/14 09:27:06 [Error] Volcado_logs - Error handling writing final pipeline state to location (non-fatal)
2026/09/14 09:27:06 [Error] Volcado_logs - org.apache.hop.core.exception.HopException:
2026/09/14 09:27:06 [Error] Volcado_logs - Error getting execution from Neo4j
2026/09/14 09:27:06 [Error] Volcado_logs - Error registering execution data to Neo4j
2026/09/14 09:27:06 [Error] Volcado_logs -
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerData(NeoExecutionInfoLocation.java:1109)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.engines.local.LocalPipelineEngine.stopTransformExecutionInfoTimer(LocalPipelineEngine.java:527)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.engines.local.LocalPipelineEngine.pipelineCompleted(LocalPipelineEngine.java:484)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.Pipeline.fireExecutionFinishedListeners(Pipeline.java:1461)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.Pipeline.lambda$startThreads$0(Pipeline.java:1259)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.transform.BaseTransform.fireTransformFinishedListeners(BaseTransform.java:2978)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.transform.BaseTransform.markStop(BaseTransform.java:2971)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:141)
2026/09/14 09:27:06 [Error] Volcado_logs - at java.base/java.lang.Thread.run(Thread.java:1583)
2026/09/14 09:27:06 [Error] Volcado_logs - Caused by: org.apache.hop.core.exception.HopRuntimeException: Error registering execution data to Neo4j
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerNeo4jData(NeoExecutionInfoLocation.java:1186)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.lambda$registerData$13(NeoExecutionInfoLocation.java:1107)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalSession.lambda$transaction$1(InternalSession.java:163)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.retry.ExponentialBackoffRetryLogic.retry(ExponentialBackoffRetryLogic.java:119)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalSession.lambda$transaction$0(InternalSession.java:160)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.observation.util.ObservationUtil.observe(ObservationUtil.java:40)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalSession.transaction(InternalSession.java:160)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalSession.execute(InternalSession.java:145)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalSession.executeWrite(InternalSession.java:123)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.Session.executeWrite(Session.java:123)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerData(NeoExecutionInfoLocation.java:1107)
2026/09/14 09:27:06 [Error] Volcado_logs - ... 8 more
2026/09/14 09:27:06 [Error] Volcado_logs - Caused by: org.apache.hop.core.exception.HopRuntimeException: Error saving rows and their metadata in Neo4j
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.saveNeo4jRowsAndMeta(NeoExecutionInfoLocation.java:1363)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerNeo4jData(NeoExecutionInfoLocation.java:1178)
2026/09/14 09:27:06 [Error] Volcado_logs - ... 18 more
2026/09/14 09:27:06 [Error] Volcado_logs - Caused by: org.neo4j.driver.exceptions.TransientException: The allocation of an extra 23.5 MiB would use more than the limit 21.0 GiB. Currently using 21.0 GiB. dbms.memory.transaction.total.max threshold reached
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.util.Futures.blockingGet(Futures.java:63)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalTransaction.lambda$run$0(InternalTransaction.java:81)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.observation.util.ObservationUtil.observe(ObservationUtil.java:40)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.InternalTransaction.run(InternalTransaction.java:80)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.AbstractQueryRunner.run(AbstractQueryRunner.java:32)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.AbstractQueryRunner.run(AbstractQueryRunner.java:37)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.DelegatingTransactionContext.run(DelegatingTransactionContext.java:41)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.execute(NeoExecutionInfoLocation.java:1747)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.saveNeo4jRowsAndMeta(NeoExecutionInfoLocation.java:1331)
2026/09/14 09:27:06 [Error] Volcado_logs - ... 19 more
2026/09/14 09:27:06 [Error] Volcado_logs - Suppressed: org.neo4j.driver.internal.util.ErrorUtil$InternalExceptionCause
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.adaptedbolt.ErrorMapper.mapToNeo4jException(ErrorMapper.java:224)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.adaptedbolt.ErrorMapper.mapBoltFailureException(ErrorMapper.java:165)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.adaptedbolt.RoutedErrorMapper.mapBoltFailureException(RoutedErrorMapper.java:64)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.adaptedbolt.ErrorMapper.map(ErrorMapper.java:81)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.adaptedbolt.AdaptingDriverResponseHandler.onError(AdaptingDriverResponseHandler.java:68)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.routed.impl.RoutedBoltConnection$RoutedResponseHandler.onError(RoutedBoltConnection.java:160)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.pooled.impl.PooledBoltConnection$PooledResponseHandler.onError(PooledBoltConnection.java:204)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.BoltConnectionImpl$ResponseHandleImpl.lambda$onError$0(BoltConnectionImpl.java:677)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.BoltConnectionImpl$ResponseHandleImpl.runIgnoringError(BoltConnectionImpl.java:809)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.BoltConnectionImpl$ResponseHandleImpl.onError(BoltConnectionImpl.java:677)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.BoltConnectionImpl$3.onError(BoltConnectionImpl.java:309)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.messaging.v3.BoltProtocolV3.lambda$run$0(BoltProtocolV3.java:431)
2026/09/14 09:27:06 [Error] Volcado_logs - at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
2026/09/14 09:27:06 [Error] Volcado_logs - at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
2026/09/14 09:27:06 [Error] Volcado_logs - at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
2026/09/14 09:27:06 [Error] Volcado_logs - at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.handlers.RunResponseHandler.onFailure(RunResponseHandler.java:50)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.async.inbound.InboundMessageDispatcher.handleFailureMessage(InboundMessageDispatcher.java:108)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.messaging.v57.MessageReaderV57.unpackFailureMessage(MessageReaderV57.java:46)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.messaging.common.CommonMessageReader.read(CommonMessageReader.java:53)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.async.inbound.InboundMessageHandler.channelRead0(InboundMessageHandler.java:81)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.async.inbound.InboundMessageHandler.channelRead0(InboundMessageHandler.java:34)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:99)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:325)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.bolt.connection.netty.impl.async.inbound.MessageDecoder.channelRead(MessageDecoder.java:40)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:361)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:348)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:470)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:288)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:355)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1429)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:176)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.handle(AbstractNioChannel.java:445)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioIoHandler$DefaultNioRegistration.handle(NioIoHandler.java:388)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioIoHandler.processSelectedKey(NioIoHandler.java:596)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioIoHandler.processSelectedKeysOptimized(NioIoHandler.java:571)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioIoHandler.processSelectedKeys(NioIoHandler.java:512)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.nio.NioIoHandler.run(NioIoHandler.java:484)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.SingleThreadIoEventLoop.runIo(SingleThreadIoEventLoop.java:225)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.channel.SingleThreadIoEventLoop.run(SingleThreadIoEventLoop.java:196)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:1195)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
2026/09/14 09:27:06 [Error] Volcado_logs - at org.neo4j.driver.internal.shaded.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2026/09/14 09:27:06 [Error] Volcado_logs - ... 1 more |
|
Your reply is again quite lacking in details :-) |
|
This feedback led to a thorough review of the Neo4j code in PR #8282 |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Hello everyone!
I'm using v2.19.0 with Neo4J v5 and I'm always getting a warning message saying that it wasn't able to register data and state into the database.
From the execution information perspective I can see the execution history and some details in each execution like logs and metrics. I've managed to know how they are working and search for errors, but I would appreciate it if I could get the execution lineage, error lineage and data tabs working.
In addition, I also noticed that many times when there is an error I can see in red the transform or action in the parent pipeline or workflow, but when I click on the drill down button to go to the referenced pipeline or workflow there is nothing marked.
So I would be grateful if someone could give me some advice of any type to try to solve this, because I'm not sure if it's something related to Apache HOP, the Neo4J configuration or anything else.
All reactions