Idea: Odysseus finetune for smaller models (1–14B), collecting samples

[pewdiepie-archdaemon]

edited to better fit the goal

Heyo,
I'm planning to start a finetune project for a dedicated Odysseus LLM.

Goal

Create a local Odysseus-specialized model that can run on more modest hardware while preserving reliable agent/tool usage.

Odysseus does not need a benchmark-maxxing model for most workflows.
It needs a fast, predictable local model that understands Odysseus
tools, emits valid tool calls, handles tool results cleanly, and
avoids unnecessary long reasoning.

Why

Current local model usage has three common failure modes:

• the model cannot handle the full Odysseus agent prompt well
• the model misunderstands tool schemas or emits invalid calls
• users pick models that are not compatible with reliable agent/tool execution

A dedicated Odysseus finetune should reduce all three by training
directly on successful Odysseus agent traces.

Model

• Qwen 3.5 9B
  This is large enough to preserve instruction-following and tool-use
  reliability, but small enough to be useful for local users once
  quantized.

• Possibly scale up or down later

Primary deployment targets:

• llama.cpp / GGUF
• vLLM
• BF16 or FP8 where appropriate
• Q4 quantized release for broad local use

Training

Phase 1: SFT.

The first goal is not RL or benchmark improvement. The first goal is
behavioral reliability:

• choose the right Odysseus tool
• produce valid arguments
• avoid unnecessary chatter
• summarize tool outputs briefly
• ask for clarification only when required
• recover from tool errors

RL can be explored later, but SFT should be enough to prove the concept.

Dataset

Target: around 20k high-quality successful Odysseus agent traces.

Generation target: possibly 80k raw traces, then filter aggressively.

The dataset should include:

• successful single-tool workflows
• multi-tool workflows
• no-tool-needed examples
• ambiguous requests
• tool failures
• empty results
• permission/scope failures
• malformed user input
• short casual user requests
• realistic long user requests

The assistant side should be compact and consistent. The model should
learn the Odysseus protocol, not learn to be verbose.

Style

The Odysseus finetune should not be a yapper.

For normal Odysseus tool usage, it should be direct:

1. understand request
2. call tool
3. read result
4. answer briefly

Long thinking is not the target. The model should be useful in
production, not optimized for public reasoning benchmarks.

User: remind me to call Ajax tomorrow
Model: calls the todo/reminder tool correctly in under a second

That is the product value.

[alteixeira20]

I like the direction, but I’d be careful not to treat finetuning as the first fix before we prove where the bottleneck really is.

If the agent prompt is already around 10K tokens, part of the problem may be architectural rather than model-specific: too much static tool/context guidance is being loaded before we know which task path the user actually needs. A finetune might help smaller models follow Odysseus conventions better, but it may also hide the fact that the prompt/tool surface itself needs to be slimmer and more modular.

My instinct would be to approach this in stages:

1. measure where the prompt tokens are actually going;
2. define a small eval suite for the major Odysseus task surfaces;
3. benchmark a few existing 1B/3B/7B/14B models against those evals;
4. try prompt slimming / per-tool or just-in-time tool guidance;
5. only then decide whether a dedicated finetune is still needed.

That would also make the dataset work much safer. If we collect samples before having evals and a clear trace format, we risk building a dataset that reflects today’s prompt shape too closely instead of the behavior we actually want Odysseus models to learn.

So I’m not against the finetune idea. I think it could be valuable, especially for self-hosted users. I’d just prefer to make the first milestone an evidence-gathering and prompt-surface reduction pass, then use that to decide whether training is the right next step or whether a lighter architecture change gets us most of the benefit.

Happy to expand on any of those areas if useful; I’m mostly thinking in terms of keeping this measurable and avoiding locking the project into the wrong fix too early.

Best regards,
Alexandre.

[pewdiepie-archdaemon]

Alexandre — you're right, and I'm with you on the split.

Two problems, two fixes:

1. Prompt bloat. ~10K tokens of static tool guidance loaded every turn regardless of what the user is actually trying to do. That's a project-side problem and the fix is to slim + modularize the surface — per-tool prompts, just-in-time loading by task path, cut redundancy. I want to do this regardless of where the finetune discussion lands. Plan: audit where the tokens actually go, cut, then test each skill/MCP usage individually before reassembling, with a small eval suite alongside so the cuts are measurable rather than vibes-based.

2. "Which model do I run?" Even with a slimmer prompt, self-hosters still hit "I have no idea which 7B actually works well with Odysseus." A finetune could answer that — but as you point out, better Cookbook recommendations + a public eval scoreboard might answer it without ever training a model. So finetune goes to backlog.

One thing I'd still want to start in parallel, lightly: sample collection. Real trace data takes weeks to accumulate, and the eval suite from (1) is going to need example traces to score against anyway. So the sample work isn't wasted even if we never train — it becomes the eval set + a public dataset for whoever does want to train. Just won't be the immediate priority.

Rough order:

• Prompt audit + cuts (measurable, near-term)
• Small eval suite, built as the cuts go in
• Benchmark existing 1B–14B candidates (Qwen3 / Llama 3 / DeepSeek / Gemma) — answer "which model to run" with data, not vibes
• Sample collection in the background, format informed by the eval suite (so we don't bake in today's prompt shape)
• Then decide if a dedicated finetune is still on the table

Thanks for the steer — exactly the kind of pushback I was hoping for when I posted this.

— Felix

[alteixeira20]

I think that split is the right way to frame it.

I opened #2750 to track the prompt-bloat side specifically. I tried to frame it as a parent roadmap/tracker rather than a giant implementation issue: measurement first, then small low-risk slimming PRs, then eval-guided modular/JIT guidance before touching anything more behavior-sensitive.

That is roughly how I interpreted your comment, with a bit of added structure around the PR order. I’d appreciate your take on whether the order/scope looks right before I start opening PRs against it.

My instinct for the first PR is still measurement-only: assembled prompt token breakdown diagnostics, with no prompt wording changes and no tool-selection changes. Once we have that baseline, the first actual slimming target looks like MCP prose filtering.

[sacb0y]

I really think there should be a debug option for the full prompt to be accessible. For me its been very hard to figure out problem areas when things fail cause I can only guess whats in the prompt.

[Zaptosis]

@pewdiepie-archdaemon look at the proposal to re-license as AGPL, its better at protecting against big tech abuses & ensuring the source code remains free.

Right now with a MIT license, OpenAI could take your projects code & implement these features into their closed source program despite it being the open source work of yourself & your community.

Based on what how you've spoke about these issues in your videos, I imagine once you look into this you'll become a full on copyleft crusader & promoting those licenses. Its such a cool legal concept that changes must remain open. Its a very powerful too to fight back against big tech & the closing off of the internet.

Also yes big models, small models, all good, very yes yes

[undergroundrap]

This makes sense to me too. Since #2750 is already tracking the prompt-bloat side, I’m happy to help with the measurement-first pass rather than jumping straight into prompt rewrites.

A focused first contribution I could take: diagnostics that report the assembled agent prompt/token breakdown by section/tool surface, with no behavior or wording changes. That would give the slimming/eval work a baseline and make later PRs easier to review.

If you already have a preferred shape for that output, I can follow it; otherwise I can propose something small and non-invasive on #2750 before opening a PR.

[alteixeira20]

That seems like a great start to me, and yes feel free to elaborate please.

[undergroundrap]

Great. I’d frame the first pass as measurement-only and explicitly non-behavioral.

What I’m thinking:

1. Add a small diagnostic path that builds the same agent prompt the runtime would send, then reports char/token breakdowns by source section.
2. Keep it read-only: no prompt wording changes, no tool-selection changes, no routing changes.
3. Make the output useful for later slimming PRs by showing which surfaces are actually expensive.

A useful breakdown might be:

• base agent preamble / rules;
• API-agent rules if present;
• builtin tool sections;
• MCP tool descriptions;
• skill index / skill snippets;
• document or active-context injection;
• integration-specific prompt additions;
• date/time/user-context preface.

For each section, I’d report something like:

• section name;
• character count;
• estimated token count using the existing estimator where available;
• percentage of total assembled prompt;
• maybe top N largest tool sections so we can see where the bloat really is.

I’d keep the first PR boring on purpose: probably a helper plus focused tests, and either a CLI/dev-only route/log output depending on what fits the existing code best. The goal would be to create a baseline that later PRs can cite before cutting or modularizing anything.

After that, the next PRs can be small and evidence-based: “this section is X% of the prompt, here is a low-risk reduction, here is the before/after token count.”

If that shape sounds right, I can write up the exact proposed output format on #2750 before touching code.

[alteixeira20]

That shape sounds right to me.

I’d keep this first pass strictly measurement-only: no prompt wording changes, no routing changes, no tool-selection changes. Just a boring diagnostic that shows where the assembled prompt budget is going, with enough section-level detail to make later slimming PRs measurable.

The breakdown you listed looks like a good starting point:

base agent preamble/rules;
API-agent additions if present;
builtin tool sections;
MCP tool descriptions;
skill index/snippets;
active document/context injection;
integration-specific additions;
date/time/user-context preface;
top largest sections/tools by estimated token count.

I think the exact output format and implementation shape should move to #2750, though, so this thread can stay focused on the broader finetune vs prompt-slimming direction. Could you add a refined version of that proposal there before opening a PR?

From my side, the ideal first PR would be small and non-invasive: helper + focused tests, producing a clear baseline that later PRs can cite when reducing or modularizing prompt sections.

[undergroundrap]

Sounds good. I’ll move the concrete diagnostic/output proposal over to #2750 and keep this thread focused on the broader direction.

I’ll keep the first implementation proposal measurement-only: helper + focused tests, with no prompt wording, routing, or tool-selection changes.

[jorgeporragas]

I want to help.

I've been using Oddyseus on a custom rig:

• Ryzen 5 5600X
• 32GB DDR4 RAM
• Radeon RX 9600 XT 16GB
• 1TB NVMe SSD
• Fedora 44

and running different tests with different agents, particularly testing it to see if it can run the workspace using my Obsidian vault, create files and manage my calendar autonomously. I've also been documenting all my findings on a personal markdown file.

I'm new to self-hosting LLM models but I'd gladly gather samples from my particular use case as long as I know the proper way to share them, I don't want to spam the forums or throw useless data at you.

[johanne-ks-hub]

I would love to see when you get Obsidian and Odysseus to work together. I unfortunalty know little to nothing about coding, but did manage to get Odysseus to work on my Mac. Would be ideal for my workflow if I could integrate my Obsidian vaults with Odysseus.

[jorgeporragas]

So far, I've had limited success, sadly. Odysseus has its own file structure, but it also includes built-in functions to read other files on your computer and index documents. Unfortunately, the AI rarely reads them correctly—or even finds them—even when given explicit file paths.

I haven't yet figured out whether this is an issue with how Odysseus is built or a limitation of the LLMs I'm using. I got Gemini 2.5 Flash running through the API, and while it's much better than Qwen 3 14B, it still occasionally struggles to see files or claims it doesn't have access to my computer. As a result, it sometimes fails to recognize the Python functions built into Odysseus.

My current assumption is that this is more of an Odysseus problem than an LLM problem. Since I'm running models locally, I have to stick to smaller models in the 7B–14B range to ensure my GPU can handle both the model and a reasonably large context window. I'll keep testing both local models and Gemini to see what I find.

If I get it working, I'll let you know!

[bitboody]

@pewdiepie-archdaemon Instead of manual logs, we should introduce a 'Trace Collector' dev-tool built into Odysseus. It would allow users to flag successful agent interactions as dataset-ready with ease. This sets a low barrier to contribution and keeps the dataset quality consistent.

— Abdelrahman

[bitboody]

If you believe this is issue-worthy, I'd be happy to create one.

[pewdiepie-archdaemon]

That would be super helpful tbh

[bitboody]

On it 🫡

[bitboody]

Done: #3060 — opened an issue for the proposal. @pewdiepie-archdaemon

[ghost]

[alteixeira20]

Don't be lonely your security concerns made me get out of bed xDD

[VykosMolt]

I think Odysseus should seriously test Ouro / Ouro-RLTT as a small-model backend.

Ouro is a looped language model, which means it gets extra effective reasoning depth by reusing latent computation over multiple internal steps before producing tokens. Parameter count undersells it badly.

Think of it like looking at a puzzle, Claude, or others like it, will look at it once and try to solve it, while Ouro will look at it once; do a part of it, look at it again and do another part and so on..

The original Ouro paper reports that the 1.4B and 2.6B models match up to 12B-class SOTA dense models across a range of benchmarks. That is the actual reason I think it belongs in this discussion: it is not merely a compressed small model, it is a different scaling direction. Dense small models mostly trade capability away to fit local hardware. Looped models spend the parameter budget differently: less static mass, more iterative latent computation.

Princetons RLTT version strengthens that further. Standard RL methods like GRPO reward only the final output/final latent state, which is a bad fit for looped models because the reasoning is distributed across the internal trajectory. RLTT instaed rewards the latent thought trajectory itself. Ouro RLTT improves substantially over GRPO on hard math/reasoning benchmarks.

I can share a Princeton/Ouro-RLTT checkpoint for testing if people are interested.

I would still separate two things: I am not claiming Ouro-RLTT automatically beats Qwen/Coder models at tool calling, file editing, or strict agent execution. Those are separate skills but for our niche, looped models are far more interesting than ordinary dense models at the same parameter count. Ouro-2.6B is closer to a 12B-class reasoning model in a 2.6B body, and the RLTT version pushes that advantage further beyond.

[VykosMolt]

Two more points which I forgot to add originally:
-Loaded Ouro 2.6B Thinking takes up around 5.7GB VRAM
-Looped models dont magically conjure up more knowledge, they do however manipulate knowledge much much more efficiently.

[pewdiepie-archdaemon]

That's really interesting, Ive never heard of Ouro. I think my goal first will be to train on a 9b ish model size. Quantized q4 should still fit most people hardware and take it from there.

[bitboody]

Status Update: I opened PR #3192 for Issue #3060 about a week ago and am aiming to have it merged sometime this week. The core backend logic is complete; the only remaining piece is frontend/UI polishing, which is currently in progress with @undergroundrap.

As a next step, I suggest we begin planning the data submission workflow so we can establish a pipeline for future model training.

[pewdiepie-archdaemon]

I'm kicking off this project now. I'll keep updating here if anyone is interested / willing to help.
Ideally I get like 30-200k samples of people who has used Odysseus data > check/filter for quality/sucess-rate. Badabing badaboom -- solid dataset for SFT.

That's likely not happening, I've used Odysseus heavily and could only extract 300 useful samples. Obviously scaling with more users would be helpful, so just throwing it out there if 10 more people here have 300 samples, thats still 3k very useful data. Please write below if you're willing to submit. Private info will obviously be stripped from training.

The plan instead meanwhile, is gathering seeds prompts first. As much as possible. See if I can then synth them to be alligned with Odysseus tools. I did a small test for this and it worked pretty well.

I have 64k sample seeds right now that are unrelated to Odysseus and will see if they can scew to Odysseus tool useage prompts. I'm expecting maybe 10% success-rate.

[pewdiepie-archdaemon]

Tried to run a 5000 sample overnight run but crashed due to not enough passing from filter, very annoyed

[pewdiepie-archdaemon]

Another crash or silent timeout at 3k, issue was dead agent streams stacking up and CPU at 100%. Shouldve realized but GPUs are cooking hard too. Idk Im in debug hell atm. Need to find rythm again hlep

[pewdiepie-archdaemon]

Ok we 5k got samples, but Im away traveling and dont want to start training (last time shit almost blew up).

Gonna keep auditing and synthing until Im back and see what I can produce

[pewdiepie-archdaemon]

Got enough samples to train now with 1.8k high quality samples after heavy filtering.
I saw Qwen came out with https://huggingface.co/Qwen/Qwen-AgentWorld-35B-A3B

I'm kinda curious to see how this performs / if accurate.
Now that I have a base sample amount I can possibly actually expand with this. Maybe I'm misinterpreting what the model is but curious to try regardless.

Still away so will likely run a training tonight if all goes well (it never does).

[pewdiepie-archdaemon]

Skipping qwen until I feel I need it.
Right now sticking iwth my goal to start training which was ... 3 days ago.

I have 2.3k samples but running through a larger model now to hopefully catch more subtle issues.

THEN train.

[pewdiepie-archdaemon]

Anyone who want to help here's how:

1. Send me handwritten prompts that has at least one use of our Odysseus related tool calls: example - “List my running model servers and tell me which one is currently the
   default.”

• “Create a note with the steps for restarting my local vLLM server.”
• Make prompts natural, messy, and specific.
• Include the intended Odysseus tool/category if you know it.
• Prefer real workflows: email, files, notes, models, research, calendar,
  tasks, docs.

2. Share successful Odysseus traces
   Most useful: exported chats where Odysseus actually used tools
   successfully.

I especially want:

• Tool call traces
• User prompt
• Tool name + arguments
• Tool output
• Final assistant answer
• Whether the result was good/bad

3. Share your Odysseus DB only if you are comfortable
   This may contain private chats, emails, paths, contacts, documents,
   tokens, or other sensitive data.

If you share a DB:

• Make a copy first.
• Remove anything private if possible.
• Tell me whether I can use it for training data.
• Tell me whether I can publish derived anonymized examples.
• Do not share it publicly. DM/upload privately only.

{
"prompt": "user request here",
"expected_tool": "optional tool name/category",
"notes": "optional context, what should happen, what makes it good"
}

[pewdiepie-archdaemon]

That's really good for the future but doesnt solve right now haha. Wish we'd maybe added it before.

I'm hesitant to having something like this in Odysseus until it's at least more clear what is needed. I think doing this first pass training will give me a better idea what it will look at.

Even though its voluntary, I feel like its a slippery slope with collecting data directly through Odysseus and I wanna be careful. Lmk if I got anything wrong from that.

[bitboody]

@pewdiepie-archdaemon
That makes total sense, and I completely respect that boundary. Preserving user trust and avoiding telemetry creep in Odysseus is definitely the right call.

Just to clarify what I was picturing—I didn't mean automated background collection! I was thinking of a completely offline flow: the user manually selects the data, our logic strips it of all secrets locally, and the user downloads a clean text file to their machine. If they choose to share it, they would have to manually attach it to a GitHub issue or send it to you out-of-band. It's still completely manual, just making the extraction process a lot faster and easier.

[pewdiepie-archdaemon]

I think I'd rather give people a script they can run separate from Odysseus that does this instead of baking it into Odysseus itself. What do you think?

[pewdiepie-archdaemon]

If you want to help work on that :) hehe

[bitboody]

@pewdiepie-archdaemon
Sure thing :D However, I'd like to know how you would like it to operate, I have two ideas for how this could go:

Option 1: The In-Memory App Wrapper (Interactive UI without core changes)
A standalone script (e.g., extract_tool.py) that acts as an in-memory wrapper. When a user runs it, it boots up Odysseus and injects the extraction endpoints and UI hooks strictly in memory—without modifying a single line of the core repository files on disk. This keeps the main codebase completely untouched, but still gives users who explicitly choose to run the script a clean, interactive interface to select their sessions locally.

Option 2: Interactive CLI
A command-line script that parses the database and interactively lists recent sessions or messages in the terminal, allowing the user to select exactly what they want to export using keyboard inputs.

I personally recommend the first option, since I think it has the easier way of extraction, the CLI option could work too but since we already have the PR changes it would be faster to go for the first option. What do you think?

[alteixeira20]

I agree with @pewdiepie-archdaemon instinct here: for this first pass, I would keep the extraction helper outside Odysseus and make it a boring interactive CLI first.

The in-memory app wrapper sounds convenient, but it also sounds like the kind of thing that can become fragile quickly: boot the app, inject endpoints, touch UI hooks, depend on current frontend/backend internals, and create a bigger trust/privacy story than we need before the data format is proven.

For now, the highest-value thing is not UI. It is a clean, reproducible export/audit contract:

• read local DB/session files only when the user points to them;
• show sessions locally and let the user select what to export;
• redact obvious secrets, paths, tokens, emails, and private identifiers locally;
• produce JSONL plus a small manifest;
• include user prompt, tool calls, arguments, tool outputs, final answer, pass/fail flag, and optional notes;
• produce a summary report: tool coverage, tool-call count, skipped destructive/time-consuming traces, suspected PII hits, empty-result traces, tool-error traces, and overuse candidates;
• never upload automatically. The user manually reviews and decides what to share.

That would also help with the problem Felix is hitting now: we need to know whether a trace is useful training signal, not just whether it exists. If a sample uses 12 tools where 1 or 2 should be enough, or if the prompt does not match the environment, the exporter should make that visible instead of treating it as dataset-ready.

So my vote would be:

1. external CLI first;
2. schema + manifest + local redaction + summary report;
3. fake/sandbox environment support next;
4. only then consider a UI wrapper if the flow proves useful.

The CLI can still be friendly and interactive, but it keeps the trust boundary clear: Odysseus is not collecting telemetry, nothing is uploaded, and users explicitly choose what leaves their machine.

[bitboody]

Makes total sense to me. Should I start a new repository separate from Odysseus since Felix wants it as an external addition? or should I include the CLI tool in a PR?

[alteixeira20]

I think it's worth to keep it in this repo, but as a standalone script making it scope clear and easy to maintain and use.

[bitboody]

@alteixeira20 @pewdiepie-archdaemon I added a small script to my PR, if it looks good to you, we can close this pull request in favour of another one where we work just on this script.

[bitboody]

[bitboody]

I think this might be the better option after all, I am still working on improving it.
I hope it might be useful all ¯_(ツ)_/¯

[pewdiepie-archdaemon]

Ok I did my first round of training and Odysseus LLM v0.01 is done.
ITS SO FUCKING FAST, I love it. my god its beautiful.
I've only tested document creation so far and if I can get every tool call to this level it's going to be awesome, Im so excited

[pewdiepie-archdaemon]

This is what Im here trying to fix lol I cant count for every model failure with vague explanation of how that happen.

--- No more trouble shooting here --

[pewdiepie-archdaemon]

Yeah still working just not much progress.
I'm seeing more upside with a dedicated model, since you can actually fix the weird harness quirks.
Been parallel experimenting with ablation to de-censor the base model. I plan for it to be de-censored but not sexual or violence compliance.

[pewdiepie-archdaemon]

My next move is email. So ANYONE please, start a conversation with a competent model -> ask it a few email related tasks (only email for now please) -> repeat 10-100 times and send to me, would be very helpful!!

[RaresKeY]

I put a compact private trace bundle here: (sent you read-only invite)

https://github.com/RaresKeY/odysseus-email-ui30-traces

Caveats: this is mostly AI-run data, not a hand-made dataset. I did 3 manual UI sanity checks, then ran 30 generated prompts through the real Odysseus UI/chat flow against a local GreenMail SMTP/IMAP setup. on top of my local built-in MCP Python/PYTHONPATH fix (PR #5117), also I only posted compact prompt/tool/final-answer traces and sumaries, raw logs are left out.

Lmk if it is useful or you need a different set

[pewdiepie-archdaemon]

Oh man this is so helpful thank you!!!!
If more people chip in the more variety it has, sp please @everyone

[pewdiepie-archdaemon]

Ok, I need more varied data for Odysseus to thrive so I made a site for users to submit. Can I get feedback on this? https://data.pewdiepie.com/

[RaresKeY]

Would it be acceptable to add a button straight into Odysseues to submit chat? where Odysseus would redact locally, show user exact bundle, and let them submit from in-app? Specifically opt-in and controlled by user.

Also a small UI bug:
The collapsible opens entire row insteda of one card:

[pewdiepie-archdaemon]

Thank you for feedback, very helpful!
I think Odysseus shouldn't directly collect data in anyway even though it would be nice heh.

The ui bug is fair but I also cba haha

[Stressed-Out09]

Hi Felix,

For us who are currently needing to run those smaller models having issues, if the goal is just to get good responses would using something like google's free tier api be worth while to get good sample data for you?

Would love to help but sadly my tool calling doesn't seem to like me too much to get a good run.

Thanks for your work on this!

p.s. my job is investigating local AI hosting - I do IT in the education sector - and we're genuinely testing out Odysseus on a mac mini!

[neil969]

Hey Felix,
Just to be clear, you only want stuff that actually worked, correct? So if I asked it to reply to an email & it THINKS it worked but didn't actually create any email, you don't want it? :)

[pewdiepie-archdaemon]

Hi Neil,

I do need some failures it helps the model acknowledge if it's wrong. But I value successful samples way more

[fredosauce]

edited to better fit the goal

Heyo, I'm planning to start a finetune project for a dedicated Odysseus LLM.

Goal

Create a local Odysseus-specialized model that can run on more modest hardware while preserving reliable agent/tool usage.

Odysseus does not need a benchmark-maxxing model for most workflows. It needs a fast, predictable local model that understands Odysseus tools, emits valid tool calls, handles tool results cleanly, and avoids unnecessary long reasoning.

Why

Current local model usage has three common failure modes:

• the model cannot handle the full Odysseus agent prompt well
• the model misunderstands tool schemas or emits invalid calls
• users pick models that are not compatible with reliable agent/tool execution

A dedicated Odysseus finetune should reduce all three by training directly on successful Odysseus agent traces.

Model

• Qwen 3.5 9B
  This is large enough to preserve instruction-following and tool-use
  reliability, but small enough to be useful for local users once
  quantized.
• Possibly scale up or down later

Primary deployment targets:

• llama.cpp / GGUF
• vLLM
• BF16 or FP8 where appropriate
• Q4 quantized release for broad local use

Training

Phase 1: SFT.

The first goal is not RL or benchmark improvement. The first goal is behavioral reliability:

• choose the right Odysseus tool
• produce valid arguments
• avoid unnecessary chatter
• summarize tool outputs briefly
• ask for clarification only when required
• recover from tool errors

RL can be explored later, but SFT should be enough to prove the concept.

Dataset

Target: around 20k high-quality successful Odysseus agent traces.

Generation target: possibly 80k raw traces, then filter aggressively.

The dataset should include:

• successful single-tool workflows
• multi-tool workflows
• no-tool-needed examples
• ambiguous requests
• tool failures
• empty results
• permission/scope failures
• malformed user input
• short casual user requests
• realistic long user requests

The assistant side should be compact and consistent. The model should learn the Odysseus protocol, not learn to be verbose.

Style

The Odysseus finetune should not be a yapper.

For normal Odysseus tool usage, it should be direct:

1. understand request
2. call tool
3. read result
4. answer briefly

Long thinking is not the target. The model should be useful in production, not optimized for public reasoning benchmarks.

User: remind me to call Ajax tomorrow Model: calls the todo/reminder tool correctly in under a second

That is the product value.

pewds -

this is fredosauce from twitter. it's been a longass time since i've touched github. but first thing: do you think a macbook pro m5 pro w/ 64GB RAM would be able to run odysseus?

my repos are all trash... practice stuff mostly. i'm not so sure if i'm even a good fit or honestly what i'm even doing but i'm unemployed and would like to contribute to something. maybe i can help w/ some aspect of design?

lmk!

-fredo (@fsczero1)

[pewdiepie-archdaemon]

yea that should be plenty to fit a decent model but no clue how well/fast it will run.

[pewdiepie-archdaemon]

Welp, got way less data than I thought haha.
data.pewdiepie.com is still there if anyone wants to contribute but for now, back to synthesizing.

[pewdiepie-archdaemon]

I got a sponsor to lend me cloud compute so could actually release a decent size model of Odysseus. I think I could be done with v1 in maybe 3 weeks. Got a good workflow now that everything is in place. Notes, Documents, works well. Moving on to Calendar now.