Change default ollama type in config

[paridhi-parajuli]

Summary 📝

The guardian-guardrails tool was using the default config from akd/configs/guardrails_config.py.

Details

Describe more what you did on changes.

1. Changed SERVER as the default ollama_type

Bugfixes 🐛 (delete if dind't have any)

Checks

• Closed #798
• Tested Changes
• Stakeholder Approval

[github-actions]

❌ Tests failed (exit code: 1)

📊 Test Results

• Passed: 288
• Failed: 3
• Warnings: 242
• Coverage: 74%

Branch: test/ollama-url
PR: #201
Commit: 72613dc9bf4f9f03b9bef410b2b472bea51f76b4

📋 Full coverage report and logs are available in the workflow run.

[NISH1001]

@TigranTigranTigran could you look and test this PR fix and see. paridhi was able to resolve something.

[NISH1001]

@paridhi-parajuli could you also run the changes code through pre-commit hoooks?

uv pip install pre-commit

pre-commit install

[github-actions]

❌ Tests failed (exit code: 1)

📊 Test Results

• Passed: 288
• Failed: 3
• Warnings: 222
• Coverage: 74%

Branch: test/ollama-url
PR: #201
Commit: 573139de4b190841b410f4a3acfca385cd49c674

📋 Full coverage report and logs are available in the workflow run.

[paridhi-parajuli]

@TigranTigranTigran
I also wanna highlight that at this part:

result = self.ollama_client.chat(
                    model="granite3-guardian:2b",
                    messages=messages,
)

I have hardcoded the model because self.model was coming granite3-guardian:8b although i set the default to be GUARDIAN_2B
That was weird, lmk you you can see something in code thats overriding the default model to 8b.

[NISH1001]

Things

[paridhi-parajuli]

In line 102, the default guardian model was set to 2b. But in line 167, if we had to use self.model instead of hardcoding the value, it would still have granite3-guardian:8b (although we set the default to 2b). This is the issue I am facing.
I hardcoded it to test it in the backend.

[TigranTigranTigran]

In line 102, the default guardian model was set to 2b. But in line 167, if we had to use self.model instead of hardcoding the value, it would still have granite3-guardian:8b (although we set the default to 2b). This is the issue I am facing. I hardcoded it to test it in the backend.

@paridhi-parajuli I've just tested the code on develop and I'm not seeing the same problems.

• right before calling chat, I see that self.model is correctly showing whatever it was set to in the config
• from ollama import chat works fine for me with OllamaType.CHAT, but you have to have pulled the model in Ollama advance (e.g., ollama run ibm/granite3.3-guardian:8b)
• to use OllamaType.SERVER, I first run OLLAMA_HOST=127.0.0.1:11435 ollama serve and it works fine

[paridhi-parajuli]

@TigranTigranTigran, I tried using the latest develop branch with the backend. It still gives me this issue:

ERROR    | akd.tools.granite_guardian_tool:_call_guardian:190 - [GuardianTool] Ollama error: Failed to connect to Ollama. Please check that Ollama is downloaded, running and accessible. https://ollama.com/download

But my ollama is running fine when i test using any curl command.
Looks like the chat doesnt take the ollama_base_url , rather sends the request to a default url (localhost:11434). You might have ollama running in that url so you're not getting that issue.
Can you try running ollama in a different port and test?

[TigranTigranTigran]

@TigranTigranTigran, I tried using the latest develop branch with the backend. It still gives me this issue:

ERROR    | akd.tools.granite_guardian_tool:_call_guardian:190 - [GuardianTool] Ollama error: Failed to connect to Ollama. Please check that Ollama is downloaded, running and accessible. https://ollama.com/download

But my ollama is running fine when i test using any curl command. Looks like the chat doesnt take the ollama_base_url , rather sends the request to a default url (localhost:11434). You might have ollama running in that url so you're not getting that issue. Can you try running ollama in a different port and test?

@paridhi-parajuli But in that case, you need to set ollama_type = OllamaType.SERVER in the config. If you do that, it won't use chat, but will instead use _ollama_server_gen(...) which should point it to the correct url.

Can you try it with ollama_type = OllamaType.SERVER and see if that works?

[paridhi-parajuli]

Had a call with @TigranTigranTigran ,
Figured out that the granite guardian tool was taking default configs from akd/configs/guardrails_config.py.
Recommended to change the default ollama type to SERVER
The backend will need to send the config for the guardrails with the SingleAgentNodeTemplate when it needs to be something other than the default config.

[github-actions]

❌ Tests failed (exit code: 1)

📊 Test Results

• Passed: 288
• Failed: 3
• Warnings: 223
• Coverage: 74%

Branch: test/ollama-url
PR: #201
Commit: 801a8c903d219bb6a9de3371d2c84eed3261565c

📋 Full coverage report and logs are available in the workflow run.

[NISH1001]

@paridhi-parajuli @TigranTigranTigran can you both verify if the working solution in the PR is the final one so that i can merge

[paridhi-parajuli]

This solution works fine with the backend.

[NISH1001]

@TigranTigranTigran can we also add proper tests for tehse so that in future we can capture this at tests level as well? Could be another PR. Some integration tests or something. Feel free to apprve this PR.