Repository navigation
Is RPC broken for llama-server? #26919
Replies: 2 comments
|
I've tried rebuilding llama.cpp multiple times already and even matched build tool versions. I'll leave this up for another few days before moving to VLLM for distributed inference. |
0 replies
|
Create an issue with a reproduction |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
I have two machines, both Windows, both built llama.cpp from source with
-DGGML_RPC=ON, both on Release b10358 and for some reason it can only successfully connect on llama-cli. I need llama-server to connect to a harness.The same command replaced with llama-cli works (I've already opened up the firewall for LAN access on the worker node). From digging through online discussions, it has clearly worked for people at some point but I cannot find anything recent, I am unsure what I am doing wrong.
All reactions