1_basic_agent.py produces inconsistent results

[analytic-garden]

1_basic_agent.py from the tutorial produces inconsistent results when run multiple times. Should the path through the graph be consistent?

The query was: "What is 20+(2*4)? Use a tool to calculate every step."

$ python 1_basic_agent.py

Running step b19cd878-82e8-43d3-9e42-ccb3fa0f29a0. Step input: What is 20+(2*4)? Use a tool to calculate every step.
Thought: The current language of the user is English. I need to use the add and multiply tools to help me calculate the expression step by step.
Action: add
Action Input: {'a': 2, 'b': 4}
Observation: 6
Running step ee5f48f2-f76a-42d1-9fb7-58ee173d7c9e. Step input: None
Thought: (Implicit) I can answer without any more tools!
Answer: Action: multiply
Action Input: {"a": 2, "b": 4}
Action: multiply
Action Input: {"a": 2, "b": 4}

$ python 1_basic_agent.py

Running step 1249e3f7-485f-4818-9578-09bde5a90dfb. Step input: What is 20+(2*4)? Use a tool to calculate every step.
Thought: The current language of the user is English. I need to use tools to help me calculate the expression step by step.
Action: multiply
Action Input: {'a': 2, 'b': 4}
Observation: 8
Running step e50dc969-faf8-4082-9afe-4607ea98b971. Step input: None
Thought: (Implicit) I can answer without any more tools!
Answer: Action: add
Action Input: {"a": 20, "b": 8}
Action: add
Action Input: {"a": 20, "b": 8}

The first example is incorrect. The first action that example should have been multiply.

[analytic-garden]

The example also doesn't print the full response:

...
Observation: 28
Thought: I can answer without using any more tools. I'll use the user's language to answer
Answer: The result of 20 + (2 * 4) is 28.
The result of 20 + (2 * 4) is 28.

as shown here: https://docs.llamaindex.ai/en/stable/understanding/agent/

[arielbeckjit]

whoever searching for it - when switching to 4o model (gpt-4o) - results started to be consistent and it's always calling all tools