praisonai insist using openai and cannot use the local model from ollama

[Themisstone]

I want to use the local model to integrate with praisonai

even if I use export OPENAI_BASE_URL=http://localhost:11434/v1
OPENAI_MODEL_NAME=deepseek-r1:14b
OPENAI_API_KEY=NA
The ollama is checked running, however
the praisonAI --init insists to use OPENAI, and always show the error as follow, sometimes it work, but most of the time it never works and insist using OPENAI API, can anyone help, many thanks

[14:14:31] INFO [14:14:31] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.436051 seconds
[14:14:32] INFO [14:14:32] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.882234 seconds
[14:14:33] INFO [14:14:33] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.459925 seconds
INFO [14:14:33] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.901005 seconds
[14:14:34] INFO [14:14:34] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.422548 seconds
[14:14:35] INFO [14:14:35] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.811394 seconds
INFO [14:14:35] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.435414 seconds
[14:14:36] INFO [14:14:36] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.820558 seconds
[14:14:37] INFO [14:14:37] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.404516 seconds
INFO [14:14:37] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.842089 seconds
[14:14:38] INFO [14:14:38] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.484937 seconds
[14:14:39] INFO [14:14:39] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.929251 seconds
[14:14:40] INFO [14:14:40] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.470558 seconds
INFO [14:14:40] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.997868 seconds
[14:14:41] INFO [14:14:41] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.400135 seconds
INFO [14:14:41] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.780188 seconds
[14:14:42] INFO [14:14:42] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.438576 seconds
[14:14:43] INFO [14:14:43] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.938123 seconds
[14:14:44] INFO [14:14:44] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.472849 seconds
INFO [14:14:44] _base_client.py:1099 INFO Retrying request to /chat/completions _base_client.py:1099
in 0.916233 seconds
Traceback (most recent call last):
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
yield
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 250, in handle_request
resp = self._pool.handle_request(req)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 256, in handle_request
raise exc from None
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py", line 236, in handle_request
response = connection.handle_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 101, in handle_request
raise exc
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 78, in handle_request
stream = self._connect(request)
^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_sync/connection.py", line 124, in _connect
stream = self._network_backend.connect_tcp(**kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_backends/sync.py", line 207, in connect_tcp
with map_exceptions(exc_map):
File "/opt/anaconda3/envs/myenv/lib/python3.11/contextlib.py", line 158, in exit
self.gen.throw(typ, value, traceback)
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 8] nodename nor servname provided, or not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1003, in _request
response = self._client.send(
^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_client.py", line 914, in send
response = self._send_handling_auth(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_client.py", line 942, in _send_handling_auth
response = self._send_handling_redirects(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
response = self._send_single_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_client.py", line 1014, in _send_single_request
response = transport.handle_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 249, in handle_request
with map_httpcore_exceptions():
File "/opt/anaconda3/envs/myenv/lib/python3.11/contextlib.py", line 158, in exit
self.gen.throw(typ, value, traceback)
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 8] nodename nor servname provided, or not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/opt/anaconda3/envs/myenv/bin/praisonai", line 8, in
sys.exit(main())
^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/praisonai/main.py", line 7, in main
praison_ai.main()
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/praisonai/cli.py", line 284, in main
self.agent_file = generator.generate()
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/praisonai/auto.py", line 130, in generate
response = self.client.chat.completions.create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/instructor/patch.py", line 143, in new_create_sync
response = retry_sync(
^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/instructor/retry.py", line 151, in retry_sync
for attempt in max_retries:
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/tenacity/_init.py", line 443, in _iter
do = self.iter(retry_state=retry_state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/tenacity/init.py", line 376, in iter
result = action(retry_state)
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/tenacity/init.py", line 418, in exc_check
raise retry_exc.reraise()
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/tenacity/init.py", line 185, in reraise
raise self.last_attempt.result()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/concurrent/futures/_base.py", line 449, in result
return self.__get_result()
^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
raise self._exception
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/instructor/retry.py", line 154, in retry_sync
response = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_utils/_utils.py", line 279, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/resources/chat/completions/completions.py", line 879, in create
return self._post(
^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1290, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 967, in request
return self._request(
^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1027, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1105, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1027, in _request
return self._retry_request(
^^^^^^^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1105, in _retry_request
return self._request(
^^^^^^^^^^^^^^
File "/opt/anaconda3/envs/myenv/lib/python3.11/site-packages/openai/_base_client.py", line 1037, in _request
raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.

[jtbwatson]

Had a similar issue - I had to set all five of these in order to get it to work locally in sequential process

export OPENAI_API_KEY=fake-key
export OPENAI_API_BASE=http://localhost:11434/v1
export OPENAI_BASE_URL=http://localhost:11434/v1
export MODEL_NAME=gemma3:4b
export OPENAI_MODEL_NAME=gemma3:4b

however, I still have the issue of it wanting to default to gpt-4o for hierarchical process that I'm trying to figure out...

[DrInfiniteExplorer]

Seems you need to add manager_llm: gemma3:4b to the yaml file

[MarRothm]

Why not specify the model to use on the agent rather than setting an environment variable.
When using ollama models I use the syntax eg. "llm= ollama/llama3.2:3b"

[xiongyw]

As I tested with praisonai v2.0.80, for use with ollama, one should set MODEL_NAME environment variable properly with a ollama/ prefix. For example, if i want to use model phi, I set as the following MODEL_NAME=ollama/phi.

btw, the web ui code, as shown in ui/code.py after line 80, the model name is save in the setting. so if you run the first time without MODEL_NAME set properly, the default one gpt-4o-mini will be used and saved in the setting. Next time even you run with a valid MODEL_NAME, it will not be used, unless you manually set the model name to empty from the UI...A bit tricky.

[xiongyw]

I want to use the local model to integrate with praisonai

even if I use export OPENAI_BASE_URL=http://localhost:11434/v1

OPENAI_MODEL_NAME=deepseek-r1:14b
OPENAI_API_KEY=NA
The ollama is checked running, however
the praisonAI --init insists to use OPENAI, and always show the error as follow, sometimes it work, but most of the time it never works and insist using OPENAI API, can anyone help, many thanks

After some troubleshooting, it seems that the following settings works for me with praisonai v2.0.80. I only use praisonai code so far.

(py312) bruin@dify ~ $ cat praisonai-code.sh
#!/bin/bash

#source ~/.bashrc
#conda activate py312


# Note:  as tested with "praisonai[code]" v2.0.80 with miniconda python 3.12 env,
# when using ollama service, only the following two env var are needed:
# - `MODEL_NAME`: prefix the model name with `ollama/`. cf <https://github.com/MervinPraison/PraisonAI/issues/394#issuecomment-2763242284>
# - `OLLAMA_API_BASE`: `http://ip:port`, no trailling `/` or `/v1` needed.


echo "MODEL_NAME=$MODEL_NAME"
echo "OLLAMA_API_BASE=$OLLAMA_API_BASE"

export MODEL_NAME='ollama/qwq:32b-q2k'
export OLLAMA_API_BASE="http://host.example.cn:11434"

echo "----------------------------------"
echo "MODEL_NAME=$MODEL_NAME"
echo "OLLAMA_API_BASE=$OLLAMA_API_BASE"


# Binding IP: cf <https://github.com/MervinPraison/PraisonAI/issues/431>
export CHAINLIT_HOST=0.0.0.0
export CHAINLIT_PORT=8086

# set this for debugging. default is "INFO"
#export LOGLEVEL=DEBUG

pushd ~/work/tmp/llama.cpp

praisonai code

popd

Btw, for a reasonable sized code base, a query from praisonai code will usually sized up to 500KiB, which means a model supporting very large context window is required (also implying a GPU with a lot of VRAM), if the query is not to be truncated. As I don't have a such a GPU yet, I cannot for now tell how good praisonai code is working with different models...My intention is to test "code understanding" capability of the setup.

[MervinPraison]

@claude review this request and check if this issue is fixed. If not implement the fix

[claude]

Claude finished @MervinPraison's task —— View job • claude/issue-394-20250528_143330 • Create PR ➔

---

Ollama Integration Issue Fixed ✅

I've successfully identified and fixed the Ollama integration issue reported in #394.

Todo List

• Read repository setup instructions (CLAUDE.md) - No CLAUDE.md found
• Analyze the current model configuration system
• Examine environment variable handling in the codebase
• Review the auto.py file mentioned in the traceback
• Identify the root cause of the Ollama configuration issue
• Check model initialization and client setup
• Test current Ollama integration behavior
• Implement fixes if issues are found
• Update documentation if needed

Root Cause & Fix ✅

Problem: The AutoGenerator class (used by praisonai --init) and PraisonAI CLI class only checked for OPENAI_API_BASE, but users commonly set OPENAI_BASE_URL (the standard OpenAI SDK environment variable).

Fix Applied:

1. Modified auto.py lines 94-112 - Added support for multiple environment variable patterns
2. Modified cli.py lines 116-137 - Applied same fix for consistency
3. Updated documentation - Comprehensive Ollama integration guide

Supported Environment Variables Now ✅

Users can now use any of these patterns:

✅ Standard OpenAI SDK (Recommended):

export OPENAI_BASE_URL=http://localhost:11434/v1
export MODEL_NAME=deepseek-r1:14b  
export OPENAI_API_KEY=NA

✅ Community Recommended:

export OLLAMA_API_BASE=http://localhost:11434
export MODEL_NAME=gemma3:4b
export OPENAI_API_KEY=fake-key

✅ Legacy (Still Works):

export OPENAI_API_BASE=http://localhost:11434/v1
export OPENAI_MODEL_NAME=llama2
export OPENAI_API_KEY=not-needed

Priority Order

• Model: MODEL_NAME > OPENAI_MODEL_NAME > "gpt-4o" (default)
• Base URL: OPENAI_BASE_URL > OPENAI_API_BASE > OLLAMA_API_BASE > "https://api.openai.com/v1" (default)

Files Changed

• src/praisonai/praisonai/auto.py - Fixed environment variable handling
• src/praisonai/praisonai/cli.py - Applied same fix for consistency
• docs/models/ollama.mdx - Updated documentation with all patterns

Testing

Created and verified comprehensive test covering all environment variable combinations mentioned in the issue comments.

Result

The original error should no longer occur. Users can now run:

export OPENAI_BASE_URL=http://localhost:11434/v1
export MODEL_NAME=deepseek-r1:14b
export OPENAI_API_KEY=NA
praisonai --init "Movie script about AI"

And it will correctly connect to their local Ollama instance instead of trying to reach OpenAI's servers.