-
Notifications
You must be signed in to change notification settings - Fork 328
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug]: multiple pipeline executors inside pipeline causes error #3503
Comments
confirmed, thanks for the detailed example and reproduction path! |
I've found the reason for the bug in the BaseTransformMeta.createTransformData() method for RecordsFromStreamMeta, so the code needs to be strengthened for this reflexive operation. |
As a workaround, you can replace the "Get records from stream" transformation with "Get Rows from Result" in sub_pipeline_2. |
usbrandon
added a commit
that referenced
this issue
Jan 27, 2024
Fix multiple pipeline executors inside pipeline causes error #3503
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Apache Hop version?
2.7.0 (2023-11-17 12.19.09)
Java version?
openjdk 11.0.20 2023-07-18 LTS
Operating system
Windows
What happened?
Priority = 1 because "important component is nonfunctional".
I set up a pipeline (parent pipeline) that runs two other pipelines (sub-pipelines) via the "Pipeline Executor" component. The first sub-pipeline will execute fine if its results are directed to a "Dummy (Do Nothing)" component. However, an error message is observed if the results are directed to the second sub-pipeline. I believe this behavior persists for any data passed from one pipeline executor to another within a parent pipeline. Here is the error:
An exactly analogous setup in Spoon (General Availability Release - 9.4.0.0-343) throws no errors. I've attached a compressed .zip file of the example: example.zip. Note that to observe the behavior you should:
example.zip
.hop-gui.bat
.../example/parent_pipeline.hpl
pipeline.sub_pipeline_1.hpl
andsub_pipeline_2.hpl
resolve correctly within the respectivePipeline Executor
components.Dummy (Do Nothing)
component - observe that there are no errors.sub_pipeline_1.hpl
andDummy (Do Nothing)
.sub_pipeline_1.hpl
andsub_pipeline_2.hpl
, choosing option =This output will contain the result rows after execution
.sub_pipeline_2.hpl
andDummy (Do Nothing)
.Dummy (Do Nothing)
component - observe the error from above.Issue Priority
Priority: 1
Issue Component
Component: Pipelines
The text was updated successfully, but these errors were encountered: