I'm running FLM serve command on laptop with Windows 11 x64 and Ryzen AI 7 360 Pro.
flm serve gemma4-it:e4b -p 1234 --ctx-len 65536 --q-len 20 --socket 20 --cors 0
I run a simple C# console application which uses OpenAI API and define tools like get_time, list_directory, read_file, write_file, edit_file as well as system prompt to use skills in .skills subdirectory. I have .skills/task-manager/SKILL.md file with the description of the skill to manage TODO list in tasks.md file.
I enter the following user prompt: I need to buy bread, eggs and milk. Add the task to the todo list.
The behavior with FLM:
You: I need to buy bread, eggs and milk. Add the task to the todo list
Model: I can certainly help you manage your tasks, but I do not have a built-in "todo list" tool or file system access to maintain one unless you specify where it should be saved.
If you would like me to add this task to a specific file (e.g., `todo.txt`), please let me know the path, and I can use the `write_file` tool for you!
The behavior with llama.cpp (the command is .\llama-server -hf ggml-org/gemma-4-E4B-it-GGUF --jinja -c 0 --host 127.0.0.1 --port 1234):
You: I need to buy bread, eggs and milk. Add the task to the todo list
→ list_directory({"path":".skills"})
← {
"directory": ".skills",
"subdirectories": [
"task-manager"
],
"files": [],
"message": "If you don\u0027t see the file you need, check subdirectories."
}
→ read_file({"path":".skills/task-manager/SKILL.md"})
← **SKILL.md content**
→ read_file({"path":"tasks.md"})
← { "error": "File not found: tasks.md" }
→ write_file({"content":"# My Tasks\n- [ ] Buy bread, eggs and milk","path":"tasks.md"})
← { "status": "success", "message": "File saved to tasks.md." }
Model: I have added "Buy bread, eggs and milk" to your todo list in `tasks.md`.
.skills/task-manager/SKILL.md file:
---
name: task-manager
description: Manages a daily todo list, adding new tasks or marking existing ones as complete with a timestamp.
---
# Task Manager Skill
You are responsible for managing the user's task list located at `tasks.md`.
## Workflow: Marking a Task as Complete
If the user asks to complete, finish, or check off a task, follow these EXACT steps:
1. **Get Context**: Call `read_file(path="tasks.md")` to see the current tasks.
* *Fallback*: If the file does not exist, tell the user there are no tasks to complete.
2. **Get Time**: Call `get_time()` so you know when this task was completed.
3. **Execute Update**: Call `edit_file` to update the task.
* `search_text`: The exact line of the uncompleted task (e.g., `- [ ] Fix the C++ memory leak`)
* `replacement_text`: The same line, but with an `x` and the current date/time appended (e.g., `- [x] Fix the C++ memory leak (Completed: Thursday, October 26)`)
## Workflow: Adding a New Task
If the user asks to add a new task, follow these EXACT steps:
1. **Check State**: Call `read_file(path="tasks.md")` to see the current file.
2. **Create if Missing**: If the file does not exist, use `write_file` to create `tasks.md` with the content:
# My Tasks
- [ ] New Task Name
3. **Append if Exists**: If the file *does* exist, use `edit_file` to add the task.
* *Hint for Appending*: Find the last task in the list to use as your `search_text`.
* `replacement_text`: The last task, followed by a newline `\n`, followed by the new task `- [ ] Your new task`.
## Constraints
- ALWAYS use `read_file` before `edit_file` so you know the exact `search_text` to use.
- Do not make up tasks; only use what the user provided.
C# source code (.NET 10):
using Microsoft.Extensions.Logging;
using System.ClientModel;
using System.ClientModel.Primitives;
using System.Reflection;
using System.Text.Json;
using OpenAI;
using OpenAI.Chat;
using OpenAI.Models;
#pragma warning disable OPENAI001
Init.LoggingOptions.EnableLogging = true;
Init.LoggingOptions.EnableMessageLogging = true;
Init.LoggingOptions.EnableMessageContentLogging = false;
OpenAIClient client = Init.CreateClient();
List<string> models = await Init.GetAvailableModels(client);
if (models.Count == 0)
{
Console.WriteLine("No models found.");
return;
}
Init.PrintModels(models);
string model = "gemma4-it:e4b"; //models[0];
Console.WriteLine($"Using model: {model}");
ChatClient chatClient = client.GetChatClient(model);
ChatCompletionOptions options = new()
{
Temperature = 0.1f, // Reduced for tool use consistency; 0.8 is often too "creative" for JSON tool calling
Tools =
{
ChatTool.CreateFunctionTool(
functionName: "get_time",
functionDescription: "Returns the current time. Use this when the user's request is time-sensitive or relates to scheduling."),
ChatTool.CreateFunctionTool(
functionName: "list_directory",
functionDescription: "Lists files/folders. REQUIRED for 'Skill Discovery': check the '.skills' folder to see what specialized skills are available.",
functionParameters: BinaryData.FromString("""
{
"type": "object",
"properties": {
"path": { "type": "string", "description": "Path to list (e.g. '.skills' or './project')" }
},
"required": ["path"]
}
""")),
ChatTool.CreateFunctionTool(
functionName: "read_file",
functionDescription: "Reads file content. REQUIRED for 'Skill Activation': read the '.skills/<name>/SKILL.md' file to learn how to perform a specific task.",
functionParameters: BinaryData.FromString("""
{
"type": "object",
"properties": {
"path": { "type": "string", "description": "Relative path to the file." }
},
"required": ["path"]
}
""")),
ChatTool.CreateFunctionTool(
functionName: "write_file",
functionDescription: "Creates a new file or overwrites an existing one with new content.",
functionParameters: BinaryData.FromString("""
{
"type": "object",
"properties": {
"path": { "type": "string", "description": "Path where the file should be saved." },
"content": { "type": "string", "description": "The full text content to write." }
},
"required": ["path", "content"]
}
""")),
// Search and Replace pattern is easier for 4B models than line numbers
ChatTool.CreateFunctionTool(
functionName: "edit_file",
functionDescription: "Updates a specific section of a file using a search-and-replace pattern.",
functionParameters: BinaryData.FromString("""
{
"type": "object",
"properties": {
"path": { "type": "string", "description": "Path to the file to edit." },
"search_text": { "type": "string", "description": "The exact string currently in the file that needs to change." },
"replacement_text": { "type": "string", "description": "The new text to put in place of the search_text." }
},
"required": ["path", "search_text", "replacement_text"]
}
"""))
},
};
const string systemPrompt =
"""
# Role and Environment
You are an autonomous AI orchestration agent running in a local environment.
Your primary function is to fulfill user requests by effectively using your available tools and
discovering specialized workflows called "Skills."
# Available Tools & Their Usage
You have access to the following tools. Never hallucinate tools outside of this list.
- `get_time`: Use this if the user asks for the current date/time, or if a task requires scheduling context.
- `list_directory`: Use this to explore folders. **CRITICAL:** Use this on the `.skills` directory if you need to discover available specialized workflows.
- `read_file`: Use this to read file contents. **CRITICAL:** Use this to read `.skills/<skill_name>/SKILL.md` files to learn how to execute a specific skill.
- `write_file`: Use this to create entirely new files or completely overwrite existing ones.
- `edit_file`: Use this to modify a specific section of an existing file using a search-and-replace text match.
# The "Skill Discovery" Workflow (Progressive Disclosure)
You do not know everything upfront. If the user asks for a complex task, a specific company workflow, or a specialized process, you MUST follow this sequence:
1. **Discover:** Call `list_directory` on the path `.skills`.
2. **Identify:** Look at the returned folder names. Choose the one that best matches the user's request.
3. **Learn:** Call `read_file` on `.skills/<chosen_folder>/SKILL.md`.
4. **Execute:** Read the instructions provided in the `SKILL.md` file and use your file t
""";
List<ChatMessage> history =
[
new SystemChatMessage(systemPrompt),
];
Console.WriteLine("Chat with the model.");
Console.WriteLine("Print 'exit' to exit.\n");
while (true)
{
Console.ForegroundColor = ConsoleColor.DarkCyan;
Console.Write("You: ");
string? input = Console.ReadLine()?.Trim();
Console.ResetColor();
if (string.IsNullOrWhiteSpace(input))
continue;
if (input == "exit")
{
Console.WriteLine("Bye!");
break;
}
history.Add(new UserChatMessage(input));
while (true)
{
ClientResult<ChatCompletion> response = await chatClient.CompleteChatAsync(history, options);
ChatCompletion message = response.Value;
history.Add(new AssistantChatMessage(message));
if (message.FinishReason == ChatFinishReason.ToolCalls)
{
foreach (ChatToolCall? call in message.ToolCalls)
{
Console.ForegroundColor = ConsoleColor.Yellow;
Console.WriteLine($" → {call.FunctionName}({call.FunctionArguments})");
Console.ResetColor();
string result = DispatchTool(call.FunctionName, call.FunctionArguments.ToString());
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine($" ← {result[..Math.Min(300, result.Length)]}");
Console.ResetColor();
history.Add(new ToolChatMessage(call.Id, result));
}
continue;
}
Console.ForegroundColor = ConsoleColor.Green;
foreach (ChatMessageContentPart contentPart in message.Content)
Console.WriteLine($"Model: {contentPart.Text}\n");
Console.ResetColor();
Console.WriteLine($"Input token count: {message.Usage.InputTokenCount}, " +
$"output token count: {message.Usage.OutputTokenCount}");
break;
}
}
static string DispatchTool(string name, string argsJson)
{
try
{
// Some local models wrap JSON in extra braces or markdown blocks
argsJson = argsJson.Trim().Trim('`').Replace("json", "");
if (argsJson.StartsWith("{{") && argsJson.EndsWith("}}"))
{
argsJson = argsJson.Substring(1, argsJson.Length - 2);
}
using JsonDocument doc = JsonDocument.Parse(argsJson);
JsonElement args = doc.RootElement;
return name switch
{
"get_time" => GetTime(),
"list_directory" => Listing(args.GetProperty("path").GetString()!),
"read_file" => ReadFile(args.GetProperty("path").GetString()!),
"write_file" => WriteFile(args.GetProperty("path").GetString()!, args.GetProperty("content").GetString()!),
"edit_file" => EditFile(
args.GetProperty("path").GetString()!,
args.GetProperty("search_text").GetString()!,
args.GetProperty("replacement_text").GetString()!),
_ => $"{{ \"error\": \"Unknown tool: {name}\" }}"
};
}
catch (Exception ex)
{
return $"{{ \"error\": \"Exception using tool {name}: {ex.Message}\" }}";
}
}
// Ensures the requested path is inside the BasePath to prevent directory traversal.
static string GetValidatedPath(string inputPath)
{
string basePath = Utils.GetWorkspacePath();
string fullPath = Path.GetFullPath(Path.Combine(basePath, inputPath));
if (!fullPath.StartsWith(basePath, StringComparison.OrdinalIgnoreCase))
{
throw new UnauthorizedAccessException(
"Access Denied: You cannot access files outside of the workspace directory.");
}
return fullPath;
}
static string GetTime()
{
DateTime now = DateTime.Now;
return $"{{ \"current_date\": \"{now:D}\", \"current_time\": \"{now:T}\", \"timezone\": \"{TimeZoneInfo.Local.DisplayName}\" }}";
}
static string Listing(string path)
{
try
{
string targetPath = GetValidatedPath(path);
if (!Directory.Exists(targetPath))
return $"{{ \"error\": \"Directory not found: {path}\" }}";
// Get relative paths to keep the model's context clean
IEnumerable<string> dirs = Directory.EnumerateDirectories(targetPath).
Select(p => Path.GetRelativePath(targetPath, p));
IEnumerable<string> files = Directory.EnumerateFiles(targetPath).
Select(p => Path.GetRelativePath(targetPath, p));
return JsonSerializer.Serialize(new
{
directory = path,
subdirectories = dirs,
files = files,
message = "If you don't see the file you need, check subdirectories.",
}, new JsonSerializerOptions { WriteIndented = true });
}
catch (Exception ex)
{
return $"{{ \"error\": \"{ex.Message}\" }}";
}
}
static string ReadFile(string path)
{
try
{
string targetPath = GetValidatedPath(path);
return File.Exists(targetPath)
? File.ReadAllText(targetPath)
: $"{{ \"error\": \"File not found: {path}\" }}";
}
catch (Exception ex)
{
return $"{{ \"error\": \"{ex.Message}\" }}";
}
}
static string WriteFile(string path, string content)
{
try
{
string targetPath = GetValidatedPath(path);
string? directory = Path.GetDirectoryName(targetPath);
if (directory != null && !Directory.Exists(directory))
Directory.CreateDirectory(directory);
File.WriteAllText(targetPath, content);
return $"{{ \"status\": \"success\", \"message\": \"File saved to {path}.\" }}";
}
catch (Exception ex)
{
return $"{{ \"error\": \"{ex.Message}\" }}";
}
}
static string EditFile(string path, string searchText, string replacementText)
{
try
{
string targetPath = GetValidatedPath(path);
if (!File.Exists(targetPath))
return $"{{ \"error\": \"File not found: {path}\" }}";
string content = File.ReadAllText(targetPath);
if (!content.Contains(searchText))
{
return
$"{{ \"error\": \"The exact search_text was not found in {path}. Use 'read_file' to confirm the exact content before editing.\" }}";
}
string newContent = content.Replace(searchText, replacementText);
File.WriteAllText(targetPath, newContent);
return $"{{ \"status\": \"success\", \"message\": \"Successfully replaced text in {path}.\" }}";
}
catch (Exception ex)
{
return $"{{ \"error\": \"{ex.Message}\" }}";
}
}
public static class Init
{
public static readonly ClientLoggingOptions LoggingOptions = new()
{
EnableLogging = true,
LoggerFactory = new MyLoggerFactory(),
EnableMessageContentLogging = true,
EnableMessageLogging = true,
MessageContentSizeLimit = int.MaxValue,
};
public static readonly OpenAIClientOptions ClientOptions = new()
{
Endpoint = new Uri("http://localhost:1234/v1"),
NetworkTimeout = TimeSpan.FromMinutes(10),
ClientLoggingOptions = LoggingOptions,
};
public static OpenAIClient CreateClient() => new(new ApiKeyCredential("lm-studio"), ClientOptions);
public static async Task<List<string>> GetAvailableModels(OpenAIClient client)
{
OpenAIModelCollection? models = (await client.GetOpenAIModelClient().GetModelsAsync())?.Value;
return models is null ? [] : models.Select(m => m.Id).ToList();
}
public static void PrintModels(List<string> models)
{
Console.WriteLine($"Available {models.Count} model(s):");
foreach (string model in models)
Console.WriteLine(model);
}
}
public sealed class MyLoggerFactory: ILoggerFactory
{
public void Dispose()
{
}
public ILogger CreateLogger(string categoryName) => new MyLogger();
public void AddProvider(ILoggerProvider provider)
{
}
}
public sealed class MyLogger: ILogger
{
public void Log<TState>(LogLevel logLevel, EventId eventId, TState state, Exception? exception,
Func<TState, Exception?, string> formatter)
{
ConsoleColor color = logLevel switch
{
LogLevel.Trace => ConsoleColor.Gray,
LogLevel.Debug => ConsoleColor.DarkGray,
LogLevel.Information => ConsoleColor.White,
LogLevel.Warning => ConsoleColor.Yellow,
LogLevel.Error => ConsoleColor.Red,
LogLevel.Critical => ConsoleColor.DarkRed,
_ => ConsoleColor.White,
};
ConsoleColor oldColor = Console.ForegroundColor;
Console.ForegroundColor = color;
object? newState = state;
if (newState is IReadOnlyList<KeyValuePair<string, object?>> stateProperties)
{
string json = string.Empty;
int entryIndex = -1;
for (int index = 0; index < stateProperties.Count; index++)
{
KeyValuePair<string, object?> entry = stateProperties[index];
if (entry is { Key: "content", Value: string jsonString })
{
json = GetFormattedJson(jsonString);
entryIndex = index;
break;
}
}
if (entryIndex >= 0)
{
FieldInfo? field = typeof(TState).GetField($"_value{entryIndex}",
BindingFlags.GetField | BindingFlags.Instance | BindingFlags.NonPublic);
field?.SetValue(newState, json);
}
}
Console.WriteLine(formatter((TState)newState!, exception));
Console.ForegroundColor = oldColor;
}
public bool IsEnabled(LogLevel logLevel) => true;
public IDisposable BeginScope<TState>(TState state)
where TState: notnull =>
new Scope();
private sealed class Scope: IDisposable
{
public void Dispose()
{
}
}
private static string GetFormattedJson(string jsonString)
{
try
{
using JsonDocument doc = JsonDocument.Parse(jsonString);
string formatted = JsonSerializer.Serialize(
doc.RootElement,
new JsonSerializerOptions { WriteIndented = true }
);
return formatted;
}
catch (JsonException)
{
return jsonString;
}
}
}
I'm running FLM serve command on laptop with Windows 11 x64 and Ryzen AI 7 360 Pro.
flm serve gemma4-it:e4b -p 1234 --ctx-len 65536 --q-len 20 --socket 20 --cors 0I run a simple C# console application which uses OpenAI API and define tools like
get_time,list_directory,read_file,write_file,edit_fileas well as system prompt to use skills in.skillssubdirectory. I have.skills/task-manager/SKILL.mdfile with the description of the skill to manage TODO list intasks.mdfile.I enter the following user prompt:
I need to buy bread, eggs and milk. Add the task to the todo list.The behavior with FLM:
The behavior with llama.cpp (the command is
.\llama-server -hf ggml-org/gemma-4-E4B-it-GGUF --jinja -c 0 --host 127.0.0.1 --port 1234):.skills/task-manager/SKILL.mdfile:C# source code (.NET 10):