Align with llama.cpp b1488

[SignalRT]

Align with the latest llama.cpp binaries.

[martindevans]

@SignalRT I've started a GitHub action run to build the binaries for all platforms (currently running here). I'll PR them into this branch once the run has finished.

[SignalRT]

@martindevans , see this other PR #241 because it includes the Intel build, and maybe should be included. It will enable to pass the test on GitHub con MacOS intel before they allow to run the test on MacOS arm.

[martindevans]

Related to that other PR, I just opened up this issue: #251

[martindevans]

I just pushed binaries into this PR. To be honest I expected to have to open up a PR into your fork, I didn't expect to be able to push them directly!

I added all the binaries from this run except for the MacOS ones. Those can be added too if you want, but I'll leave it up to you.

[SignalRT]

@martindevans I added the MacOS binaries from the same run and passed all the test manually successfully.

[lexxsoft]

In response of @martindevans #245 (comment) request, I have checked out https://github.com/SignalRT/LLamaSharp/tree/Align-Current-Binaries and build them. Then I have referenced LLamaSharp.dll and libllama-cuda12.dll on my test code, but I am getting

TypeInitializationException: The type initializer for 'LLama.Native.NativeApi' threw an exception.

while if I reference libllama.dll everything is fine, so I do not know how to properly test Cuda12 backend.
What else am I missing?

[martindevans]

What else am I missing?

You probably need to rename libllama-cuda12.dll to libllama.dll

[lexxsoft]

@martindevans ok, I did not realize this is p/invoke. Rename helped.

version	time
v0.5.1 Cuda12	14.696
v0.7.0 Cuda12	1:09.663
https://github.com/SignalRT/LLamaSharp/tree/Align-Current-Binaries Cuda12	12.972

And as a bonus the newest version is consistently faster than v0.5.1.

[martindevans]

Thanks for all your help with testing @lexxsoft ❤️

@SignalRT With that testing done I think this is ready to merge?

[AsakusaRinne]

After this PR has been merge, is everything about binary ok to publish a new release? Because of the performance problem in 0.7.0, I think we should add a release 0.7.1.

[martindevans]

If possible can we wait for the Intel macos PR (#258) because that will act as a test case for the runtime feature detection.

[AsakusaRinne]

If possible can we wait for the Intel macos PR (#258) because that will act as a test case for the runtime feature detection.

Sure :)

[lexxsoft]

@SignalRT , @martindevans, has 0.8.0 Cuda12 been tested on Windows?
When I try to add LLamaSharp.Backend.Cuda12 to the project I am getting Warning: No compatible assemblies found in package 'LLamaSharp.Backend.Cuda12'. Maybe this is only me?

[martindevans]

I believe @AsakusaRinne tested them?

[lexxsoft]

I believe @AsakusaRinne tested them?

Hm, this is fine on Rider, however LINQPad 7 returns that Warning: No compatible assemblies found in package 'LLamaSharp.Backend.Cuda12', fails to install the package, and throws runtime exception of missing backend.

[AsakusaRinne]

I tested with VS and it worked fine. Have you ever had problem when using previous versions of LLamaSharp with LinqPad?

[lexxsoft]

@AsakusaRinne, no, older package versions work just fine