[Feature Request] Achieve dynamic flow connections with logic conditions

[zhanghanduo]

Currently the node construction operates on a rudimentary fixed DAG. Is it possible to incorporate functionalities such as "if-else" and "for-loop" logics and branches (or even directed graph with cycles), to introduce dynamism into the workflow, thereby opening the door to boundless creative possibilities?

[thy09]

Hi @zhanghanduo , with the latest version of vscode and pip package, you can use "activate" feature to support if-else, if "activate" is set, the node will be executed only when activate condition is met. We are preparing a document to show the usage, but not ready yet.

[zhanghanduo]

Thanks for the information 👍. I will check it out.

[PeiwenGaoMS]

Hi @zhanghanduo , you can review the sample flow conditional-flow-for-if-else to understand how to use the activate config. We're also working on a more detailed document for this feature.

[dunalduck0]

Hi @PeiwenGaoMS, nice work about adding "activate" and "skip". A comment on the example. In practice, it's more likely to use "activate" when output is NOT None (instead of True/False). This is because if the output value is True/False, then the next component, who takes this output as input, is not getting much information to start with. I think "skip" could be more practical.

Another question. If a flow is skipped in the middle, what to do about these flow output that have no value assigned, because part of the flow is not run?

[dunalduck0]

@PeiwenGaoMS "activate" does not always work. It worked in your example flow, but was not effective in my flow for a LLM node.

It runs regardless the target value in activate config is "true" or "false" and always generates output.

[jiaochenlu]

Hi @dunalduck0, Can you create a separate bug issue for the above problem?

[PeiwenGaoMS]

Hi @PeiwenGaoMS, nice work about adding "activate" and "skip". A comment on the example. In practice, it's more likely to use "activate" when output is NOT None (instead of True/False). This is because if the output value is True/False, then the next component, who takes this output as input, is not getting much information to start with. I think "skip" could be more practical.

Another question. If a flow is skipped in the middle, what to do about these flow output that have no value assigned, because part of the flow is not run?

Hi @dunalduck0,
For the first question, you can let the upstream node output a dictionary to solve, for example:

{
  "is_activate": true,
  "valid_info": "...",
}

The node's activation config is then set to

activate:
    when: ${node.output.is_activate}
    is: true

For your second question, if an input to a node using a python tool references a bypassed node, you should give that input a default value.

Btw, we are preparing the doc about activate config. You can take a look, it may solve some of your issues. Thanks.
[Docs] Add doc for condition flow with activate

[PeiwenGaoMS]

@PeiwenGaoMS "activate" does not always work. It worked in your example flow, but was not effective in my flow for a LLM node.

It runs regardless the target value in activate config is "true" or "false" and always generates output.

@dunalduck0 , Please take a screenshot containing the output of non_empty_property node.

[dunalduck0]

Thank you @PeiwenGaoMS on the tip of using dictionary output. I saw it in the document as well and it worked. That's a great feature. Thus, I no longer have the "non_empty_property" node. I didn't know I could set default value for node input too (I know I can for ${inputs}. Will give it a try later.

Two feedback:

1. I didn't use the dictionary output correctly at the first try because I overlooked the importance of type hints. I even filed a bug at https://github.com/microsoft/promptflow/issues/568 but resolved myself once I found my own bug. I think it's helpful to emphasize the type hints in the document.
2. In my run, if all inputs of a node are from bypassed nodes, the node is automatically bypassed. This is fine as long as the node output is not referenced by ${outputs}; otherwise, PF will end in error. I had to add a Python node "generate_final_answer" at the end to ensure nothing referenced by ${outputs} is bypassed.

[PeiwenGaoMS]

@dunalduck0 Thank you for your feedback.

One thing I want to clarify is, the default value refers to the default value of the parameter in the python code. You can check the generate_response node in the example flow conditional-flow-for-switch.

[NishitHada]

@PeiwenGaoMS This might be related to the original comment, so posting it here.

Let's say I want to loop a llm tool over an array of questions, with a separate request for each of them. How do I do that? Or in a more general manner, how do I implement any kind of looping in promptflow?

If I try to create a cycle of tools with an activate condition to break out of the loop, I get this error-
Flow test failed with NodeCircularDependency: Invalid node definitions found in the flow graph. Node circular dependency has been detected among the nodes in your flow. Kindly review the reference relationships for the nodes ['get_hyde_questions', 'hyde', 'sem_search_answers_for_questions'] and resolve the circular reference issue in the flow.

[dunalduck0]

I did loop of embedding inside a python component. I copied the PF code for embedding but changed it to loop. Alternatively, I guess you can wire code to generate flow.diag.yaml programmatically to simulate loop.

[NishitHada]

@dunalduck0 Sorry, I didn't mention it earlier, but I'm using VS code extension. How do I copy the equivalent PF code for any tool?

[dunalduck0]

Yes, I am using VsCode extension too, but I don't know if it matters.

In a flow, you can use different tools, such as built-in "LLM" or "Python". They are all based on Python code in this repo. The built-in LLM tool takes in only one prompt. If you want it to take in a list of prompts, you can go to https://github.com/microsoft/promptflow/blob/main/src/promptflow-tools/promptflow/tools/aoai.py, copy the function "chat()", and create a new Python tool with a loop on chat().

But I don't know your question on circle. I've never seen a circle of tools in my life.

[NishitHada]

Thanks @dunalduck0, this works 🙇

[jaskhalsa]

Any updates on this?

[PeiwenGaoMS]

Hi @jaskhalsa, it looks like all the previously mentioned issues have been resolved. Do you have any other questions? If not, can I close this question?

[nikawang]

I've got the same requirements. Indeed, we can add loop/iteration logic to python code. But once we move to us Prompt Flow, we want take advantages of the capabilites.. Looking forward Prompt Flow can provide such functionality.

[brendenpetersen]

Thanks @dunalduck0, this works 🙇

@NishitHada Can you share a bit about your solution and use case here? If you loop within a single Python code, then seems like you lose out on being able to control/track the intermediate steps.

I'm interested in using planners (e.g., Semantic Kernel's SequentialPlanner) or ReAct agents, where there are a variable number of calls involved, and still being able to inspect individual outputs. There are some articles (here) on using Prompt Flow with such agents, but seems like you lose out on some major advantages of Prompt Flow like being able to see and tune intermediate outputs.

I've got the same requirements. Indeed, we can add loop/iteration logic to python code. But once we move to us Prompt Flow, we want take advantages of the capabilites.. Looking forward Prompt Flow can provide such functionality.

@nikawang Exactly. If we loop within the Python node we lose most of what Prompt Flow provides (see the article I linked above). But it seems like loops go against the design principles of Prompt Flow--a DAG with loops is no longer a DAG...

[dunalduck0]

What are your “intermediate steps”? As far as I can see, there is an HTTP call between a textual prompt and an GPT output and there is no intermediate steps. Well, unless I misunderstood the question.

[brendenpetersen]

What are your “intermediate steps”? As far as I can see, there is an HTTP call between a textual prompt and an GPT output and there is no intermediate steps. Well, unless I misunderstood the question.

The LLM nodes are just an HTTP call between a text prompt and LLM output with no intermediate steps, agreed. But here we are talking about a Python node that performs multiple LLM calls via a for loop, ReAct style prompting, an "agent" like Semantic Kernel's SequentialPlanner, etc. In these cases, it would be nice to inspect intermediate steps.

[github-actions]

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!