Implemented batch inference support #110

Merged

patrickfleith merged 2 commits intopatrickfleith:mainfrom

LamQam:feature/batch-inference

Sep 19, 2025

Commits on Sep 16, 2025

Implemented batch inference support in the generate method using LiteLLM and created tests for batch prompts and message inputs
la2032
committed

Commits on Sep 19, 2025

Merge branch 'main' into feature/batch-inference
patrickfleith
committed