.Net: Integrate LLamaSharp to enable LLaMa model inference

[AsakusaRinne]

Hi all, great work so far! As the open-source LLM models become more and more hot, will semantic-kernel consider supporting some of them such as LLaMa? I'm the main author of LLamaSharp, which is the .NET binding of llama.cpp. I wonder if it's possible to integrate LLamaSharp into semantic-kernel to add support for some open-source LLM models (as you can see, llama.cpp support 10 kinds of models). I'd like to work for this integration if it's taken into account to the future version of semantic-kernel. Thank you for your consideration.

[craigomatic]

Thanks for posting, this does sound interesting and I wonder if it would be possible to provide a LLamaSharp implementation along the lines of this sample: https://github.com/microsoft/semantic-kernel/blob/main/samples/dotnet/kernel-syntax-examples/Example16_CustomLLM.cs

@dluc make have opinions here also

[evchaki]

@AsakusaRinne , let us know if you want to do a PR for this based on the other examples of how other models work.

[AsakusaRinne]

Hi, thanks to the efforts of community, there's an example here to use LLamaSharp with sk. The example uses the latest pre-release of sk and is easy to deploy. Please have a look and I'll improve it with your advice.

[AsakusaRinne]

@evchaki @craigomatic Please let me know if I can provide any further information to help move this forward. Thank you for your time and attention.

[drasticactions]

We have a basic implementation of this in LLamaSharp itself now: https://github.com/SciSharp/LLamaSharp/tree/master/LLama.SemanticKernel

I'm not sure how stable the Abstractions library is though, I've had to bump it due to a recent change in IChatCompletion. I'm not sure bringing this implementation into this repo makes sense, but maybe there could be more said for changes like these?

[dluc]

I'm definitely going to take this in somehow :-)

[KSemenenko]

I can do this =)

[matthewbolanos]

@drasticactions, now that the abstractions have settled down, would you be interested in bringing in LlamaSharp into the main Semantic Kernel repo?

[xbotter]

I am currently working on the maintenance of LLamaSharp.SemanticKernel. If possible, I would be willing to do it.

[markwallace-microsoft]

All .Net issues prior to 1-Dec-2023 are being closed. Please re-open, if this issue is still relevant to the .Net Semantic Kernel 1.x release. In the future all issues that are inactive for more than 90 days will be labelled as 'stale' and closed 14 days later.

[AsakusaRinne]

It seems that this issue is still under-working. Please reopen it. :)