Skip to content

Model and Tool Calling

Jump to bottom
skobeltsyn edited this page Mar 28, 2026 · 6 revisions

Model Configuration and Tool Calling

Connect your agents to LLMs, define tools they can call, and understand the agentic loop that drives autonomous reasoning.

The model {} Block

Every agent that needs LLM reasoning configures it through the model {} DSL block. The block produces a ModelConfig and, under the hood, a ModelClient that speaks HTTP to the LLM provider.

val agent = agent<String, String>("researcher") {
    model {
        ollama("qwen2.5:7b")   // model name on the Ollama server
        host = "localhost"      // default
        port = 11434            // default
        temperature = 0.7       // default; lower = more deterministic
    }
    // ...
}

ModelConfig

ModelConfig is a plain data class -- no magic:

data class ModelConfig(
    val name: String,
    val provider: ModelProvider,
    val temperature: Double = 0.7,
    val host: String = "localhost",
    val port: Int = 11434,
    val client: ModelClient? = null   // override for testing
)

The ModelBuilder DSL maps directly to these fields:

DSL Function / Property ModelConfig Field Purpose
ollama("name") name, provider Selects the Ollama provider and model
host = "..." host Ollama server hostname
port = 1234 port Ollama server port
temperature = 0.3 temperature Sampling temperature
client = myClient client Inject a custom or mock ModelClient

ModelClient

ModelClient is a fun interface -- a single abstract method:

fun interface ModelClient {
    fun chat(messages: List<LlmMessage>): LlmResponse
}

The framework ships OllamaClient, which implements ModelClient by POSTing to Ollama's /api/chat endpoint. You never need to instantiate it manually; the model {} block handles it. But you can replace it entirely for testing (see the last section).

LlmMessage and LlmResponse

Messages flowing in and out of the LLM use two simple types:

data class LlmMessage(
    val role: String,          // "system", "user", "assistant", "tool"
    val content: String,
    val toolCalls: List<ToolCall>? = null
)

LlmResponse is a sealed type with two variants:

sealed interface LlmResponse {
    data class Text(val content: String) : LlmResponse
    data class ToolCalls(val calls: List<ToolCall>) : LlmResponse
}

When the LLM wants to call a tool, it returns ToolCalls. When it has a final answer, it returns Text. The agentic loop reacts accordingly.

Defining Tools

Tools are functions the LLM can invoke during its reasoning loop. You define them inside a skill's tools {} block.

The tool() DSL

skills {
    skill<String, String>("calculator", "Performs arithmetic") {
        tools("add", "multiply")   // marks this skill as agentic

        tool("add", "Add two numbers") { args ->
            val a = args["a"] as Double
            val b = args["b"] as Double
            a + b
        }

        tool("multiply", "Multiply two numbers") { args ->
            val a = args["a"] as Double
            val b = args["b"] as Double
            a * b
        }
    }
}

Each tool() call creates a ToolDef:

class ToolDef(
    val name: String,
    val description: String,
    val executor: (Map<String, Any?>) -> Any?
)

The name and description are sent to the LLM as part of the system prompt so it knows which tools exist and what they do. The executor lambda runs on the JVM when the LLM calls the tool.

Tool Arguments

Arguments arrive as Map<String, Any?> parsed from the LLM's JSON output. Common patterns:

tool("search", "Search the web") { args ->
    val query = args["query"] as String
    val maxResults = (args["max_results"] as? Number)?.toInt() ?: 10
    searchService.search(query, maxResults)
}

Because LLMs can produce unexpected types (a number as a string, a missing key), defensive casting is good practice. For structured error handling, see Tool Error Recovery.

Agentic Skills

A skill becomes agentic -- meaning the LLM drives it -- the moment you call tools("toolName") inside the skill definition:

skill<String, String>("analyze", "Analyze data using tools") {
    tools("fetch_data", "summarize")   // <-- this makes it agentic
    // ...tool definitions...
}

Without tools(...), a skill is deterministic: it runs the implementedBy lambda directly, no LLM involved. With tools(...), the skill enters the agentic loop described below.

The Agentic Loop

The agentic loop is the engine behind every LLM-driven skill. Here is what executeAgentic does step by step:

Step 1 ─ Build System Prompt
    Concatenate: agent prompt + skill description + tool list (names & descriptions)

Step 2 ─ Add User Input
    Append the user's input as a "user" message

Step 3 ─ Call LLM chat()
    Send the full message list to the ModelClient

Step 4 ─ Handle Response
    If LlmResponse.Text  ──→ parse the content, return the result. Done.
    If LlmResponse.ToolCalls ──→ go to Step 5

Step 5 ─ Execute Tool Calls
    For each ToolCall in the response:
        - Find the matching ToolDef by name
        - Run its executor with the arguments
        - Append the result as a "tool" message

Step 6 ─ Loop
    Go back to Step 3 with the updated message list

Step 7 ─ Budget Check
    Before each iteration, check maxTurns.
    If exceeded, throw BudgetExceededException.

Each iteration of steps 3-6 is one "turn." The Budget Controls page explains how to cap these.

ToolCall

The LLM produces tool calls as structured objects:

data class ToolCall(
    val name: String,
    val arguments: Map<String, Any?>
)

The framework matches name against your registered ToolDef instances and passes arguments to the executor.

Knowledge as Lazy Tools

In agentic skills, knowledge entries are exposed to the LLM as callable tools. The LLM decides when to fetch them -- it does not receive all knowledge upfront.

val agent = agent<String, String>("support-bot") {
    model { ollama("qwen2.5:7b") }

    skills {
        skill<String, String>("answer", "Answer support questions") {
            tools("lookup_faq")

            knowledge("faq", "Frequently asked questions") {
                loadText("/data/faq.txt")
            }

            knowledge("policies", "Company policies") {
                loadText("/data/policies.txt")
            }

            tool("lookup_faq", "Search the FAQ database") { args ->
                val query = args["query"] as String
                faqDatabase.search(query)
            }
        }
    }
}

When the LLM calls a knowledge tool, the onKnowledgeUsed hook fires. See Observability Hooks for details.

The Calculator Example

A complete, runnable agent with arithmetic tools:

val calculator = agent<String, String>("calculator") {
    prompt = "You are a calculator. Use the provided tools to compute the answer."

    model {
        ollama("qwen2.5:7b")
        temperature = 0.1   // low temperature for deterministic math
    }

    budget { maxTurns = 5 }

    skills {
        skill<String, String>("compute", "Perform calculations") {
            tools("add", "subtract", "multiply", "divide")

            tool("add", "Add two numbers: a + b") { args ->
                val a = (args["a"] as Number).toDouble()
                val b = (args["b"] as Number).toDouble()
                a + b
            }

            tool("subtract", "Subtract two numbers: a - b") { args ->
                val a = (args["a"] as Number).toDouble()
                val b = (args["b"] as Number).toDouble()
                a - b
            }

            tool("multiply", "Multiply two numbers: a * b") { args ->
                val a = (args["a"] as Number).toDouble()
                val b = (args["b"] as Number).toDouble()
                a * b
            }

            tool("divide", "Divide two numbers: a / b") { args ->
                val a = (args["a"] as Number).toDouble()
                val b = (args["b"] as Number).toDouble()
                require(b != 0.0) { "Division by zero" }
                a / b
            }
        }
    }

    onToolUse { name, args, result ->
        println("Tool: $name($args) = $result")
    }
}

// Usage
val answer = calculator("What is (3 + 5) * 2?")
// Tool: add({a=3, b=5}) = 8.0
// Tool: multiply({a=8.0, b=2}) = 16.0
// answer: "The result of (3 + 5) * 2 is 16."

Notice the flow: the LLM decides to call add first, gets 8.0, then calls multiply with 8.0 and 2, gets 16.0, and finally returns a text answer. Two turns, two tool calls, one final answer.

Testing with Mock ModelClient

ModelClient is a fun interface, so you can replace it with a lambda. This means you can test agents without a running Ollama server.

Basic Mock

val mockClient = ModelClient { messages ->
    // Always return a text response
    LlmResponse.Text("42")
}

val agent = agent<String, String>("test-agent") {
    model {
        ollama("unused")        // model name is irrelevant for mocks
        client = mockClient     // injected via the DSL
    }
    skills {
        skill<String, String>("answer", "Give an answer") {
            tools("lookup")
            tool("lookup", "Look something up") { args -> "data" }
        }
    }
}

val result = agent("question")
assertEquals("42", result)

Simulating Tool Calls

To test the full agentic loop, return ToolCalls on the first call and Text on the second:

var callCount = 0

val mockClient = ModelClient { messages ->
    callCount++
    when (callCount) {
        1 -> LlmResponse.ToolCalls(
            listOf(ToolCall("lookup", mapOf("query" to "test")))
        )
        else -> LlmResponse.Text("Final answer based on lookup")
    }
}

Asserting on Messages

Capture the messages the agent sends to the LLM:

val capturedMessages = mutableListOf<List<LlmMessage>>()

val mockClient = ModelClient { messages ->
    capturedMessages.add(messages.toList())
    LlmResponse.Text("done")
}

agent("input")

// Verify system prompt was constructed correctly
val systemMsg = capturedMessages[0].first { it.role == "system" }
assertTrue(systemMsg.content.contains("lookup"))

// Verify user input was passed
val userMsg = capturedMessages[0].first { it.role == "user" }
assertEquals("input", userMsg.content)

This pattern lets you write fast, deterministic unit tests for agent behavior without any LLM infrastructure. Combine with Observability Hooks for even richer test assertions.

Next Steps

Agents.KT Wiki

Project Links

Getting Started

Core Concepts

Composition Operators

LLM Integration

Guided Generation

Agent Memory

Reference

Contributing

Clone this wiki locally