Skip to content

fix(ik-llama-cpp): adapt to common_grammar struct in sampling.h #9512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mudler merged 1 commit into from
Apr 23, 2026
+12 −3
Merged

fix(ik-llama-cpp): adapt to common_grammar struct in sampling.h #9512

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backend/cpp/ik-llama-cpp/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

IK_LLAMA_VERSION?=d4824131580b94ffa7b0e91c955e2b237c2fe16e
IK_LLAMA_VERSION?=286ce324baed17c95faec77792eaa6bdb1c7a5f5
LLAMA_REPO?=https://github.com/ikawrakow/ik_llama.cpp

CMAKE_ARGS?=
Expand Down
13 changes: 11 additions & 2 deletions backend/cpp/ik-llama-cpp/grpc-server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -686,7 +686,16 @@ struct llama_server_context
slot->sparams.mirostat_eta = json_value(data, "mirostat_eta", default_sparams.mirostat_eta);
slot->params.n_keep = json_value(data, "n_keep", slot->params.n_keep);
slot->sparams.seed = json_value(data, "seed", default_sparams.seed);
slot->sparams.grammar = json_value(data, "grammar", default_sparams.grammar);
{
// upstream changed common_params_sampling::grammar from std::string to
// the common_grammar struct (type + grammar). The incoming JSON still
// carries a plain string, so build the user-provided grammar here and
// fall back to the server default when the request omits it.
std::string grammar_str = json_value(data, "grammar", std::string());
slot->sparams.grammar = grammar_str.empty()
? default_sparams.grammar
: common_grammar{COMMON_GRAMMAR_TYPE_USER, std::move(grammar_str)};
}
slot->sparams.n_probs = json_value(data, "n_probs", default_sparams.n_probs);
slot->sparams.min_keep = json_value(data, "min_keep", default_sparams.min_keep);
slot->sparams.grammar_triggers = grammar_triggers;
Expand Down Expand Up @@ -1232,7 +1241,7 @@ struct llama_server_context
// {"logit_bias", slot.sparams.logit_bias},
{"n_probs", slot.sparams.n_probs},
{"min_keep", slot.sparams.min_keep},
{"grammar", slot.sparams.grammar},
{"grammar", slot.sparams.grammar.grammar},
{"samplers", samplers}
};
}
Expand Down
Loading