[BUG]: Agent flow model problem

[ctcanbol]

How are you running AnythingLLM?

Docker (remote machine)

What happened?

I am using the latest version of AnythingLLM on Ubuntu 22.04 with Docker. The LLM for my workspace is llama3.1:8b, system LLM is Deepseek-r1:8b. I created an agent flow to call an API. At first i did not specify any custom LLM for agent calling inside the workspace. And got below error. Then tried setting workspace agent's model manually to llama3.1:8b as same as the workspace model, get the same error. Agent runs and founds the right flow, runs the API but somehow triggers OpenAI LLM, which set nowhere.

[backend] info: [AgentHandler] Attached flow flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865 (json-engine-api api call) plugin to Agent cluster
[backend] info: [AgentLLM - llama3.1:8b] Valid tool call found - running flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865.
[backend] info: [AgentHandler] [debug]: @agent is attempting to call `flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865` tool
[backend] info: [AgentHandler] LLM processing failed: The OPENAI_API_KEY environment variable is missing or empty; either provide it, or instantiate the OpenAI client with an apiKey option, like new OpenAI({ apiKey: 'My API Key' }). Error: The OPENAI_API_KEY environment variable is missing or empty; either provide it, or instantiate the OpenAI client with an apiKey option, like new OpenAI({ apiKey: 'My API Key' }).
     at new OpenAI (/app/server/node_modules/openai/index.js:53:19)
     at new OpenAIProvider (/app/server/utils/agents/aibitat/providers/openai.js:51:20)
     at AIbitat.getProviderForConfig (/app/server/utils/agents/aibitat/index.js:755:16)
     at executeLLMInstruction (/app/server/utils/agentFlows/executors/llm-instruction.js:28:30)
     at FlowExecutor.executeStep (/app/server/utils/agentFlows/executor.js:86:24)
     at FlowExecutor.executeFlow (/app/server/utils/agentFlows/executor.js:127:35)
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async AgentFlows.executeFlow (/app/server/utils/agentFlows/index.js:157:12)
     at async Object.handler (/app/server/utils/agentFlows/index.js:213:30)
     at async AIbitat.handleExecution (/app/server/utils/agents/aibitat/index.js:634:22)

1/8 Agent @agent invoked. Swapping over to agent chat. Type /exit to exit agent execution loop early.
2/8 [debug]: @agent is attempting to call `flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865` tool
3/8 Executing flow: json-engine-api api call
4/8 Making POST request to external API...
5/8 Sending body to http://172.16.5.63:31292/json-engine-api/v1/retrieve-json-data-editor: {"screenCode":"gdpr"}
6/8 API call completed
7/8 Processing data with LLM instruction...
8/8 Sending request to LLM...
9/9 Flow failed: Unknown error

Are there known steps to reproduce?

I can reproduce the issue on my environment.

[timothycarambat]

What is your System LLM preference and what is your Workspace agent model (if set at all)
cc: @shatfield4

[ctcanbol]

System:

Workspace:

Workspace Agent:

P.S: with this configuration and setting workspace agent to default system does not matter; both returns same error.

[ctcanbol]

Thanks for the quick fix!

Now after using latest version, the error message changed:

 [backend] info: [AgentLLM - llama3.1:8b] Valid tool call found - running flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865.
 [backend] info: [AgentHandler] [debug]: @agent is attempting to call `flow_78fcd5ca-aab0-49e4-8fed-e7a79f473865` tool
 [backend] info: [AgentHandler] [AgentFlowToolExecutor] - executing API Call block
 [backend] info: [AgentHandler] [AgentFlowToolExecutor] - executing LLM Instruction block
 [backend] info: [AgentHandler] Sending request to LLM (ollama::llama3.1:8b)
 [backend] info: [AgentHandler] LLM processing failed: Cannot read properties of null (reading 'length') TypeError: Cannot read properties of null (reading 'length')
     at OllamaProvider.complete (/app/server/utils/agents/aibitat/providers/ollama.js:49:21)
     at executeLLMInstruction (/app/server/utils/agentFlows/executors/llm-instruction.js:34:39)
     at FlowExecutor.executeStep (/app/server/utils/agentFlows/executor.js:87:24)
     at FlowExecutor.executeFlow (/app/server/utils/agentFlows/executor.js:129:35)
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async AgentFlows.executeFlow (/app/server/utils/agentFlows/index.js:151:12)
     at async Object.handler (/app/server/utils/agentFlows/index.js:202:30)
     at async AIbitat.handleExecution (/app/server/utils/agents/aibitat/index.js:634:22)
     at async AIbitat.reply (/app/server/utils/agents/aibitat/index.js:578:21)
     at async AIbitat.chat (/app/server/utils/agents/aibitat/index.js:374:15)