Add async call for LlamaCpp

[ninely]

Add async call and async stream for LlamaCpp.
This allows it to use async callback handler to return a stream response.
@hwchase17, @rlancemartin

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
langchain	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 20, 2023 10:21am

[faridelya]

Thanks alot @ninely.

[ninely]

@nfcampos Thank you for your feedback. It's greatly appreciated. I've made some adjustments based on your suggestions.

[wdoppenberg]

Is there a reason this hasn't been merged yet? Any help needed?

[hamaadtahiir]

why is it not merged yet?

[prometx11]

I am also looking forward for a merge, need this feature!
Are there any outstanding changes?

[akionux]

@ninely Thanks for your work. To work with langserve (#12441), I additionally added chunk=chunk, but llamacpp.py has been moved to langchain_community, so I applied your changes to new llamacpp.py in my branch.

[baskaryan]

Apologies for the slow review! Pr has some merge conflicts, happy to re-review if you'd like to resolve

[nfcampos]

Thanks, this delegation to the sync stream method is now done by the base class BaseChatModel so this functionality already exists, closing.