Loading state when awaiting llm responses

[hunter547]

I was noticing that backroad can only support synchronous llm responses, which is not a realistic scenario given llm requests don't provide immediate replies. This one was tough to implement and not sure this is the best strategy, but it gets the job done and looks great on the UI. Open to make changes, as I'm sure I didn't follow a pattern or two. Thanks

[hunter547]

[sudo-vaibhav]

Thanks for the PR, give me some time to go through the code changes and get back with my feedback

[hunter547]

@sudo-vaibhav Sounds good, thanks

[hunter547]

@sudo-vaibhav Checking in to see if you’ve had a chance to review this PR. Thanks!

[sudo-vaibhav]

This is a nice idea, and I appreciate this contribution. I see two ways ahead -

1. This code as you showed in the screenshot works. And I understand the value that this will add immediately to the LLM experience. And here I propose diving slightly deeper and seeing if we can enhance this further.

2. The real core ask which I understand here is a better way of handling blocking actions across backroad. Maybe a standardized way or experience of handling UI when part of it is blocked via an await. Here in LLMs, it manifests itself as a loader in the message box, but the clean programmer in me (or whatever is left of that disillusioned old man within me) thinks of if we can come up with a core change that allows adding these loading experiences across the board for more than just this use case.

Maybe this second idea will create a subpar experience for individual components (maybe the maximum that we can achieve with this generic approach is putting a loader on the whole LLM container as a whole) but then cater to a much broader array of use cases.

Since you have worked much more on this interesting problem and have done the cool hands-on work while implementing this LLM loader, do you have any inputs on how we might implement this loading concept once framework-wide, and then be able to make these individual component implementations of loaders in much fewer lines of code @hunter547

[sudo-vaibhav]

lgtm for now, merging to alpha, can prioritise refactoring later

[sudo-vaibhav]

Good work @hunter547 , urge you to also raise a PR on the docs repo, so that people can be aware of how to use this new addition here: https://github.com/sudomakes/backroad-documentation/blob/main/src/content/docs/docs/API%20Reference/LLM.mdx