LlamaSharp.ToolCallEnvelopes

LlamaSharp.ToolCallEnvelopes gives a local LlamaSharp model one small JSON language for final answers, tool requests, and optional refusals. A compiled plan keeps the prompt, GBNF grammar, parser, and argument validator together, so the model cannot be prompted for one shape and parsed as another.

The package targets .NET 10 and leaves model loading, sampling, native context ownership, and chat-template selection in your LlamaSharp adapter.

Install

dotnet add package Supprocom.LlamaSharp.ToolCallEnvelopes --version 0.2.0

Managed control flow

Managed control flow is the short path for an application that wants a useful tool loop today. The runner creates turns, retries invalid model output, validates every call before dispatch, executes calls sequentially, appends tool results, and stops on a final answer or refusal.

var weather = ToolDefinition.Parse(
    "get_weather",
    "Gets the current weather for one city.",
    """
    {
      "type": "object",
      "properties": {
        "city": { "type": "string", "minLength": 1, "maxLength": 64 },
        "unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
      },
      "required": ["city", "unit"],
      "additionalProperties": false
    }
    """);

var plan = ToolEnvelopePlan.Compile([weather]);

var result = await ToolEnvelopeRunner.RunAsync(
    executor,
    plan,
    "Use current weather data when the user asks about weather.",
    [ToolMessage.User("What is the weather in Zagreb?")],
    async (call, cancellationToken) =>
    {
        var city = call.Arguments.GetProperty("city").GetString();
        return await GetWeatherJsonAsync(city!, cancellationToken);
    },
    cancellationToken: cancellationToken);

if (result is ToolRunResult.Completed
    {
        Outcome: ToolEnvelopeOutcome.AssistantMessage answer
    })
{
    Console.WriteLine(answer.Text);
}

The executor implements ILlamaSharpToolExecutor. It applies the model's native chat template to turn.Prompt, attaches turn.Grammar with the root start rule, and yields only newly generated text. The complete working LlamaSharp adapter is in the demo.

Manual control flow

Manual control flow exposes the same compiled turn without choosing retries, dispatch policy, history storage, context reuse, or when the next turn should start. It does not require ILlamaSharpToolExecutor or ToolEnvelopeRunner. Existing application code may produce the model response through ChatSession, ChatAsync, InferAsync, or any other inference path; manual TCE code only creates the exact turn and parses its output.

var conversation = new List<ToolMessage>
{
    ToolMessage.User("What is the weather in Zagreb?"),
};

var turn = plan.CreateTurn(
    "Use current weather data.",
    conversation,
    ToolChoice.Auto);

// Existing application code produces this response using turn.Prompt and
// turn.Grammar. No TCE executor interface is involved.
string output = existingModelOutput;
var outcome = turn.Parse(output);

The host still maps turn.Prompt through the model's native chat template and attaches turn.Grammar with start rule root. It keeps its existing inference, retry, dispatch, history, and persistence code. The complete manual turn, streaming, and tool-loop examples are in manual control.

Read getting started for the managed path, manual control for host-owned orchestration, and the schema profile for the exact accepted JSON Schema subset.