Feat/ollama integration

[srijanpatel]

Add support for models hosted by ollama

This PR adds support for connecting self-hosted models using ollama with PySpur. The ollama endpoint has to be specified in the .env file before launching the PySpur service.

Summary of changes:

• use ollama python sdk to generate responses for ollama models. Litellm could not be used as it currently has an unresolved bug with JSON mode JSON output throws error for ollama provider BerriAI/litellm#6353
• centralise env management
• minor clean up of key management API and settings modal

---

Important

Integrates Ollama models, centralizes environment management, updates key management APIs, and adjusts Docker and frontend for these changes.

• Ollama Integration:
  • Added support for Ollama models using ollama Python SDK in llm_utils.py.
  • Introduced OllamaOptions class for API call options.
  • Added ollama_with_backoff() function for API calls with retry logic.
• Environment Management:
  • Centralized environment variable management in key_management.py.
  • Updated .env.example with Ollama configuration.
• Key Management API:
  • Refactored key management in key_management.py to use MODEL_PROVIDER_KEYS.
  • Updated API endpoints to handle Ollama keys.
• Docker and Entrypoint:
  • Modified docker-compose.yml to include .env file and extra hosts.
  • Added test_ollama.sh script to verify Ollama connection in entrypoint.sh.
• Frontend Changes:
  • Updated SettingsModal.tsx and api.ts to handle API keys more effectively.
  • Adjusted API calls to reflect backend changes.

^{This description was created by for 8a5f886. It will automatically update as commits are pushed.}

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to 7e44fa5 in 2 minutes and 6 seconds

More details

• Looked at 448 lines of code in 10 files
• Skipped 0 files when reviewing.
• Skipped posting 5 drafted comments based on config settings.

1. backend/app/nodes/llm/llm_utils.py:19

• Draft comment:
  Ensure litellm.set_verbose=True is a valid usage. If set_verbose is not a valid attribute or method, this line should be removed or corrected.
• Reason this comment was not posted:
  Comment did not seem useful.

2. backend/app/nodes/llm/llm_utils.py:20

• Draft comment:
  load_dotenv() is called multiple times across different files. Consider centralizing this call to avoid redundancy.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The load_dotenv() function is called multiple times across different files. This is unnecessary and can be centralized to avoid redundancy.

3. backend/app/nodes/llm/llm_utils.py:193

• Draft comment:
  Ensure consistent handling of api_base. Check for non-empty values before using it to avoid unexpected behavior.
• Reason this comment was not posted:
  Comment did not seem useful.

4. backend/app/nodes/llm/single_llm_call.py:17

• Draft comment:
  load_dotenv() is called multiple times across different files. Consider centralizing this call to avoid redundancy.
• Reason this comment was not posted:
  Confidence changes required: 50%
  The load_dotenv() function is called multiple times across different files. This is unnecessary and can be centralized to avoid redundancy.

5. frontend/src/utils/api.ts:258

• Draft comment:
  Update listApiKeys to return response.data instead of response.data.keys to match the backend response format.
• Reason this comment was not posted:
  Comment looked like it was already resolved.

Workflow ID: wflow_bC91faIyjfRF7mCK

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on ac6d9c5 in 31 seconds

More details

• Looked at 13 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. backend/app/nodes/llm/llm_utils.py:285

• Draft comment:
  The docstring mentions parameters (response_model, max_retries, initial_wait, max_wait) that are not in the function signature. Please update the docstring to reflect the actual parameters.
• Reason this comment was not posted:
  Comment was on unchanged code.

Workflow ID: wflow_0zfuDA4MNdNL9smj

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 162192c in 23 seconds

More details

• Looked at 13 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 1 drafted comments based on config settings.

1. backend/app/nodes/llm/llm_utils.py:284

• Draft comment:
  The retry attempts have been increased from 1 to 3, which aligns with the retry logic used in other functions like completion_with_backoff. This change should improve reliability.
• Reason this comment was not posted:
  Confidence changes required: 0%
  The change in retry attempts from 1 to 3 in the ollama_with_backoff function is consistent with the retry logic used in other functions like completion_with_backoff. This change is likely intended to improve reliability.

Workflow ID: wflow_UdbByDj9hSNBQ7bb

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 8a5f886 in 12 seconds

More details

• Looked at 55 lines of code in 1 files
• Skipped 0 files when reviewing.
• Skipped posting 0 drafted comments based on config settings.

Workflow ID: wflow_IRKFF4AsKwrmZ3sC

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.