Feature Request - LM Studio Streaming

[ckep1]

Love this plugin so far! Streaming works well with Ollama, but I'd love to be able to use this with LM Studio to enable streaming on Windows too with local LLMs. I have found LM Studio server setup to be much easier than Ollama, more choices are provided and it works on Windows.

Since it has an Open AI style API, I have seemingly been able to get it working both with the LocalAI and OpenAI base link settings under the Advanced Tab. This is great, but I need to wait for inferencing to finish before I get output from LM Studio.

Streaming does work otherwise, I believe all the API request needs is "stream": true added on the LM Studio side.

Not sure how much work is needed to handle the streaming within the plugin itself, I tried to make this change quickly to the API calls in a fork but completions vs streaming seem differently configured and therefore more work would be required.

Obsidian Copilot has this implemented for reference.

Thanks for the plugin - great work!

[longy2k]

Let me know if you can get LM Studio to stream on v1.8.2, thanks!

https://github.com/longy2k/obsidian-bmo-chatbot/wiki/How-to-setup-with-LM-Studio

[ckep1]

Amazing! Got it working with streaming!

Thank you for the super fast response here, great new update overall.

Had no expectation this would be working anytime soon, let alone same day. Cheers!