Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Does Dora-Dataflow not support multiple Dynamic inputs? #607

Closed
chengzi0103 opened this issue Aug 2, 2024 · 4 comments
Closed

Does Dora-Dataflow not support multiple Dynamic inputs? #607

chengzi0103 opened this issue Aug 2, 2024 · 4 comments
Labels
bug Something isn't working python Python API

Comments

@chengzi0103
Copy link

Describe the bug
When I use multiple Dynamic nodes in a Dataflow, the Dataflow cannot capture the Dynamic inputs after starting (no response). Also, when I run python dynamic.py, there is no response.

I would like to ask, is this a bug or the current design, or is the Dynamic structure designed this way? Will this issue be fixed in future versions?

@github-actions github-actions bot added bug Something isn't working python Python API labels Aug 2, 2024
@Hennzau
Copy link
Collaborator

Hennzau commented Aug 2, 2024

Describe the bug

When I use multiple Dynamic nodes in a Dataflow, the Dataflow cannot capture the Dynamic inputs after starting (no response). Also, when I run python dynamic.py, there is no response.

I would like to ask, is this a bug or the current design, or is the Dynamic structure designed this way? Will this issue be fixed in future versions?

Hi! Thanks for the report,

  • what is exactly the version of Dora you are using ? crates.io 0.3.5, main branch ? Fix tag ?

  • can you give us the exact dataflow description (YAML) ?

  • can you give us the python code of your dynamic nodes ?

@chengzi0103
Copy link
Author

Describe the bug
When I use multiple Dynamic nodes in a Dataflow, the Dataflow cannot capture the Dynamic inputs after starting (no response). Also, when I run python dynamic.py, there is no response.
I would like to ask, is this a bug or the current design, or is the Dynamic structure designed this way? Will this issue be fixed in future versions?

Hi! Thanks for the report,

  • what is exactly the version of Dora you are using ? crates.io 0.3.5, main branch ? Fix tag ?
  • can you give us the exact dataflow description (YAML) ?
  • can you give us the python code of your dynamic nodes ?

@Hennzau

  1. We are using version 3.5 of dora-rs, and the version is the main branch.
  2. We used the dynamic method for nodes with node_id names web_search_task and reasoner_task_input.
  3. task_input.py is our Python code.
nodes:
  - id: web_search_task
    path: dynamic
    inputs:
      direction: dora/timer/secs/1
    outputs:
    - task
  - id: web_search_agent
    operator:
      python: ./scripts/web_search_agent.py
      inputs:
        web_search_task: web_search_task/task
      outputs:
      - web_search_results
      - web_search_resource
      - web_search_aggregate_output
  - id: more_question_agent
    operator:
      python: ./scripts/more_question_agent.py
      inputs:
        web_search_aggregate_output: web_search_agent/web_search_aggregate_output
      outputs:
      - more_question_results
      - web_search_aggregate_output
  - id: web_search_output
    operator:
      python: ./scripts/web_search_output.py
      inputs:
        web_search_aggregate_output: more_question_agent/web_search_aggregate_output
      outputs:
      - web_search_output
      - web_search_results
      - web_search_resource
      - more_question_results
      - web_search_task
  - id: reasoner_task_input
    path: dynamic
    inputs:
      direction: dora/timer/secs/1
    outputs:
    - reasoner_task
  - id: reasoner_agent
    operator:
      python: ./scripts/reasoner_agent.py
      inputs:
        reasoner_task: reasoner_task_input/reasoner_task
        more_question_results: more_question_agent/more_question_results
      outputs:
      - reasoner_result
  - id: reasoner_output
    operator:
      python: ./scripts/reasoner_output.py
      inputs:
        reasoner_result: reasoner_agent/reasoner_result
      outputs:
      - reasoner_output

Apart from different node_ids, other results of the dynamic nodes are the same.

import json

from dora import Node
import pyarrow as pa

node = Node("web_search_task")

event = node.next()
task_data = input('Please enter your task:  ',)
node.send_output('task',pa.array([json.dumps(task_data)]),event["metadata"])

@Hennzau
Copy link
Collaborator

Hennzau commented Aug 15, 2024

Hi, sorry for the long time between my last response. Did you succeed in receiving multiple inputs?

@chengzi0103
Copy link
Author

Hi, sorry for the long time between my last response. Did you succeed in receiving multiple inputs?

Thank you so much, we have successfully resolved the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working python Python API
Projects
None yet
Development

No branches or pull requests

3 participants