Skip to content

Conversation

@psriramsnc
Copy link
Collaborator

@psriramsnc psriramsnc commented Nov 14, 2025

🚀 SyGra works with LiteLLM!

Bringing more flexibility, more model support, and smoother integrations 🎉🤖

🧩 Summary

This PR integrates LiteLLM into SyGra, enabling seamless use of OpenAI, Azure OpenAI, and VLLM models through a unified backend.
It adds shiny new model wrappers ✨, updates the model factory ⚙️, introduces new tests 🧪, improves documentation 📚, and cleans up configuration logic 🧹.

⚙️ Features Implemented

  • ✨ Added LiteLLM-backed model implementations (OpenAI, Azure OpenAI, VLLM)
  • 🔧 Updated model factory to support backend: litellm
  • 📦 Added LiteLLM dependency in pyproject.toml
  • 🧪 Added comprehensive unit tests for LiteLLM-backed models
  • 📚 Updated documentation for model configuration
  • 🧹 Cleanup of imports, validation logic, and config parsing

🚄 Performance Impact

  • 📉 No direct performance metrics included
  • 🌐 LiteLLM may add slight routing overhead depending on setup
  • 🟰 Overall: N/A — no measurable performance impact reported

🧪 How to Test

To verify the functionality end-to-end:

  1. ▶️ Run any SyGra flow or LLM node using models with: openai, azure_openai, vllm.
    LiteLLM is now the default backend for these models. Ensure requests route through LiteLLM and outputs look correct.
  2. 🔄 Test backward compatibility by setting backend: default in models.yaml.
    The pipeline should use the original custom models, not LiteLLM.
  3. 🧪 Run make test-verbose to validate all new test cases are passing successfully.

📸 Screenshots (if applicable)

N/A 🖼️

✅ Checklist

  • 🧹 Lint fixes and unit testing done
  • 🔁 End-to-end task testing
  • 📝 Documentation updated

🗒️ Notes

🎯 No behavioral changes for end users.
This update is fully backward compatible, and all existing flows will continue to work exactly as before.

@psriramsnc psriramsnc self-assigned this Nov 14, 2025
@psriramsnc psriramsnc added the enhancement New feature or request label Nov 14, 2025
@psriramsnc psriramsnc marked this pull request as ready for review November 17, 2025 06:35
@psriramsnc psriramsnc requested a review from a team as a code owner November 17, 2025 06:35
amitsnow
amitsnow previously approved these changes Nov 20, 2025
Copy link
Collaborator

@amitsnow amitsnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

zephyrzilla
zephyrzilla previously approved these changes Nov 20, 2025
Copy link
Member

@zephyrzilla zephyrzilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@psriramsnc psriramsnc dismissed stale reviews from zephyrzilla and amitsnow via f3cdfdb November 20, 2025 10:37
Copy link
Collaborator

@amitsnow amitsnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀

@psriramsnc psriramsnc merged commit eb92fc5 into main Nov 21, 2025
6 checks passed
@psriramsnc psriramsnc deleted the scratch/litellm_integration branch November 21, 2025 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants