[Bug]: Workflow executor causes exception saving Execution Information

[enricomariam42]

Apache Hop version?

SNAPSHOT-20221128

Java version?

openjdk version "11.0.9.1" 2020-11-04 OpenJDK Runtime Environment (build 11.0.9.1+1-post-Uos-1deb10u2) OpenJDK 64-Bit Server VM (build 11.0.9.1+1-post-Uos-1deb10u2, mixed mode, sharing)

Operating system

Linux

What happened?

The bug can be reproduced in the "samples" project by running the pipeline

transforms/workflow-executor-basic.hpl

Step to reproduce:

1. Open project "samples"
2. Define a Execution data profile "last_100" with sampler "last output rows" and size "100"
3. Define a Execution information location (e.g. "neo4j" as neo4j location)
4. Define a Pipeline run configuration (e.g. "local") choosing up the previous Execution information location "neo4j" and Execution data profile "last_100"
5. Define a Workflow run configuration (e.g. "local") (optional: choose the previous Execution information location)
6. Run the workflow-executor-basic pipeline

Otherwise:

1. copy in your already configured (test) project:

transforms/workflow-executor-basic.hpl
transforms/workflow-executor-child.hwf

2. Run the workflow-executor-basic pipeline

Note:
The child process adds to result the filename under HOP_UNIT_TESTS_FOLDER, which is somewhere hardcoded. There must be some files in that folder.

log:

2022/11/30 15:59:47 - Hop - Pipeline opened.
2022/11/30 15:59:47 - Hop - Launching pipeline [workflow-executor-basic]...
2022/11/30 15:59:47 - Hop - Started the pipeline execution.
2022/11/30 15:59:48 - workflow-executor-basic - Executing this pipeline using the Local Pipeline Engine with run configuration 'local'
2022/11/30 15:59:48 - workflow-executor-basic - Execution started for pipeline [workflow-executor-basic]
2022/11/30 15:59:48 - Data for parameters.0 - Finished processing (I=0, O=0, R=0, W=3, U=0, E=0)
2022/11/30 15:59:48 - workflow-executor-child - Start of workflow execution
2022/11/30 15:59:48 - workflow-executor-child - Starting action [Wait for]
2022/11/30 15:59:49 - workflow-executor-child - Starting action [Add filenames to result]
2022/11/30 15:59:49 - workflow-executor-child - Finished action [Add filenames to result] (result=[true])
2022/11/30 15:59:49 - workflow-executor-child - Finished action [Wait for] (result=[true])
2022/11/30 15:59:49 - workflow-executor-child - Workflow execution finished
2022/11/30 15:59:49 - workflow-executor-child.hwf.0 - Workflow duration : 1.089 seconds [  1.088" ]
2022/11/30 15:59:49 - workflow-executor-child - Workflow duration : 1.089 seconds [  1.088" ]
2022/11/30 15:59:49 - workflow-executor-child - Start of workflow execution
2022/11/30 15:59:49 - workflow-executor-child - Starting action [Wait for]
2022/11/30 15:59:50 - workflow-executor-basic - Warning: unable to register execution info (data and state) at location neo4j(non-fatal)
2022/11/30 15:59:50 - workflow-executor-child - Starting action [Add filenames to result]
2022/11/30 15:59:50 - workflow-executor-child - Finished action [Add filenames to result] (result=[true])
2022/11/30 15:59:50 - workflow-executor-child - Finished action [Wait for] (result=[true])
2022/11/30 15:59:50 - workflow-executor-child - Workflow execution finished
2022/11/30 15:59:50 - workflow-executor-child.hwf.0 - Workflow duration : 1.042 seconds [  1.042" ]
2022/11/30 15:59:50 - workflow-executor-child - Workflow duration : 1.042 seconds [  1.042" ]
2022/11/30 15:59:50 - workflow-executor-child - Start of workflow execution
2022/11/30 15:59:50 - workflow-executor-child - Starting action [Wait for]
2022/11/30 15:59:51 - workflow-executor-child - Starting action [Add filenames to result]
2022/11/30 15:59:51 - workflow-executor-child - Finished action [Add filenames to result] (result=[true])
2022/11/30 15:59:51 - workflow-executor-child - Finished action [Wait for] (result=[true])
2022/11/30 15:59:51 - workflow-executor-child - Workflow execution finished
2022/11/30 15:59:51 - workflow-executor-child.hwf.0 - Workflow duration : 1.074 seconds [  1.074" ]
2022/11/30 15:59:51 - workflow-executor-child - Workflow duration : 1.074 seconds [  1.074" ]
2022/11/30 15:59:51 - workflow-executor-child.hwf.0 - Finished processing (I=0, O=0, R=3, W=5895, U=0, E=0)
2022/11/30 15:59:51 - Results.0 - Finished processing (I=0, O=0, R=3, W=3, U=0, E=0)
2022/11/30 15:59:51 - Result files.0 - Finished processing (I=0, O=0, R=5892, W=5892, U=0, E=0)
2022/11/30 15:59:51 - workflow-executor-basic - Pipeline duration : 3.41 seconds [  3.410" ]
2022/11/30 15:59:51 - workflow-executor-basic - ERROR: Error handling writing final pipeline state to location (non-fatal)
2022/11/30 15:59:51 - workflow-executor-basic - ERROR: org.apache.hop.core.exception.HopException: 
2022/11/30 15:59:51 - workflow-executor-basic - Error getting execution from Neo4j
2022/11/30 15:59:51 - workflow-executor-basic - org.apache.hop.core.exception.HopValueException: 
2022/11/30 15:59:51 - workflow-executor-basic - Unexpected conversion error while converting value [ExecutionTime Integer(15)] to an Integer
2022/11/30 15:59:51 - workflow-executor-basic - class java.lang.String cannot be cast to class java.lang.Long (java.lang.String and java.lang.Long are in module java.base of loader 'bootstrap')
2022/11/30 15:59:51 - workflow-executor-basic - 
2022/11/30 15:59:51 - workflow-executor-basic - 
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerData(NeoExecutionInfoLocation.java:878)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.engines.local.LocalPipelineEngine.stopTransformExecutionInfoTimer(LocalPipelineEngine.java:486)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.engines.local.LocalPipelineEngine.pipelineCompleted(LocalPipelineEngine.java:465)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.Pipeline.firePipelineExecutionFinishedListeners(Pipeline.java:1343)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.Pipeline.lambda$startThreads$0(Pipeline.java:1142)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.transform.BaseTransform.fireTransformFinishedListeners(BaseTransform.java:2782)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.transform.BaseTransform.markStop(BaseTransform.java:2771)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:142)
2022/11/30 15:59:51 - workflow-executor-basic - 	at java.base/java.lang.Thread.run(Thread.java:834)
2022/11/30 15:59:51 - workflow-executor-basic - Caused by: java.lang.RuntimeException: org.apache.hop.core.exception.HopValueException: 
2022/11/30 15:59:51 - workflow-executor-basic - Unexpected conversion error while converting value [ExecutionTime Integer(15)] to an Integer
2022/11/30 15:59:51 - workflow-executor-basic - class java.lang.String cannot be cast to class java.lang.Long (java.lang.String and java.lang.Long are in module java.base of loader 'bootstrap')
2022/11/30 15:59:51 - workflow-executor-basic - 
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.saveNeo4jRowsAndMeta(NeoExecutionInfoLocation.java:1116)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerNeo4jData(NeoExecutionInfoLocation.java:945)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.lambda$registerData$11(NeoExecutionInfoLocation.java:876)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.neo4j.driver.internal.InternalSession.lambda$transaction$4(InternalSession.java:150)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.neo4j.driver.internal.retry.ExponentialBackoffRetryLogic.retry(ExponentialBackoffRetryLogic.java:103)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.neo4j.driver.internal.InternalSession.transaction(InternalSession.java:146)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.neo4j.driver.internal.InternalSession.writeTransaction(InternalSession.java:124)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.neo4j.driver.internal.InternalSession.writeTransaction(InternalSession.java:118)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.registerData(NeoExecutionInfoLocation.java:876)
2022/11/30 15:59:51 - workflow-executor-basic - 	... 8 more
2022/11/30 15:59:51 - workflow-executor-basic - Caused by: org.apache.hop.core.exception.HopValueException: 
2022/11/30 15:59:51 - workflow-executor-basic - Unexpected conversion error while converting value [ExecutionTime Integer(15)] to an Integer
2022/11/30 15:59:51 - workflow-executor-basic - class java.lang.String cannot be cast to class java.lang.Long (java.lang.String and java.lang.Long are in module java.base of loader 'bootstrap')
2022/11/30 15:59:51 - workflow-executor-basic - 
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.core.row.value.ValueMetaBase.getInteger(ValueMetaBase.java:2347)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.core.row.value.ValueMetaInteger.getNativeDataType(ValueMetaInteger.java:40)
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.neo4j.execution.NeoExecutionInfoLocation.saveNeo4jRowsAndMeta(NeoExecutionInfoLocation.java:1082)
2022/11/30 15:59:51 - workflow-executor-basic - 	... 16 more
2022/11/30 15:59:51 - workflow-executor-basic - Caused by: java.lang.ClassCastException: class java.lang.String cannot be cast to class java.lang.Long (java.lang.String and java.lang.Long are in module java.base of loader 'bootstrap')
2022/11/30 15:59:51 - workflow-executor-basic - 	at org.apache.hop.core.row.value.ValueMetaBase.getInteger(ValueMetaBase.java:2260)
2022/11/30 15:59:51 - workflow-executor-basic - 	... 18 more
2022/11/30 15:59:51 - workflow-executor-basic - Execution finished on a local pipeline engine with run configuration 'local'

Issue Priority

Priority: 2

Issue Component

Component: Execution Information

[mattcasters]

Caused by BaseTransform firing the row written events for targeted rows with putRowTo(). This shouldn't happen.