Crash at download

[toxincekkk]

Hello there,

I am trying to setup this brutal thing. But everytime I try to download any model it just crash.
Is there any fix to that or is it like global problem ?
Already tried to reinstall it but nothing happened.

Thanks a lot !!!

Detected hardware
NVIDIA GeForce RTX 2060
×
6.0 / 15.9 GB RAM
×
12 cores
×
cuda
×

Odysseus Cookbook crash report

Please review this report for secrets before posting it publicly.

Task

• ID: cookbook-3f2c4db5
• Type: download
• Status: crashed
• Model/repo: nvidia/Qwen3.6-35B-A3B-NVFP4
• Host: local
• Started: 2026-06-02T09:52:32.618Z

Last captured output

(no captured output)

[CoolJohn-lab]

Known issue on the Windows build. Use Ollama for now instead of trying to host with Cookbook, or create a dual-boot Linux partition if you want to try out the cool stuff. Wait for a few days for this to get sorted. I believe a proper Windows installer is coming.

[DWIND07]

Don't we already have a proper Windows installer? I am not very familiar with this stuff, but I set up Odysseus through the manual windows guide. And, I am having the same issue as the this guy. I can't download a single model through the cookbook.

[wolfgangDGS]

It's caused by the translation layer between docker and the engine.
That's why @CoolJohn-lab suggested downloading and running models through ollama on the host machine instead of through Odysseus.

With ollama running, open a CMD terminal and input ollama run <model name>

Look up the models in the ollama website to be sure of the model name

Refresh Odysseus and you should see it

[hunlight5781-droid]

problem still persists on amd patch fork https://github.com/leo-ml-larp/odysseus-amd-patch

anyone might know whats the issue?

[AnnoDomine]

1. Try to remove the broken downloads if the files are persistent on your hard drive.
2. Check the size of the Qwen2.5-Coder:3B model and verify your free hard drive space.
3. Qwen3.6:35B will never run on your machine. The RTX 2060 only has 6 GB of VRAM, and the additional 16 GB of system memory is way too little for this model.

[hunlight5781-droid]

what model can u suggest to even run locally? or msot of decent ones would need some kind of MCP server to work properly? I got 9070xr 16gb vram and 64 ram. I tried quite a few already (not in odysseus cant get it to work), and most answer one-two question and then breaks.

[AnnoDomine]

what model can u suggest to even run locally? or msot of decent ones would need some kind of MCP server to work properly? I got 9070xr 16gb vram and 64 ram. I tried quite a few already (not in odysseus cant get it to work), and most answer one-two question and then breaks.

I am using qwen3-coder:30b with ollama in my vscode continue extension.
It works extrem well and performant.
But i splitted the model with ollama to only use 5-6 GB of VRAM from my 4070ti.
I have by myself a 4070ti with 12 GB VRAM and 64 GB DDR5 in additional with a Ryzen 9 7900.
With long tasks i never run into problems.
I have to say i do not use Odysseus as it is not interesting for me.
To identify a model i would give you the hint to show not on the params.
The math behind it is the hidden layers.
So when you select a model you can splitt into GPU and CPU useage by splitting by the hidden layers.
The model i use have 64 hidden layers.
So you can reduce in ollama the amount of layers written to the GPU:
Mine is splitted to 20 num_gpu which means 20 layers are written to the GPU from the 18GB model size in total.
18 GB / 64 Layers * 20 Layers = 5.6 GB to the VRAM. The rest is written to memory and CPU.
More layers in the GPU = faster response but context needs space too. So it can be crash.
So i would not suggest a model for you but with this information i think you can select a model which fits for you :)

[hunlight5781-droid]

sounds a bit overwhelming Im sorry xD very interesting though. the issue I had is probably because I tried to run full fledge models (small and bigger ones) as per suggestions in the internet and it was a pain either million years per answer, or pure crashing mid "thoughts". could I maybe ask you how did u exacly set up everything to work. and onem ore thing if I may ask, does agent works fully on your side? can it also lookup project files inside attached folders ect or thats smth only mcp server can do?

[AnnoDomine]

No problem buddy.
I've really cut down on using API models like Claude Code or Gemini, and now I run almost everything in my local environment. Sometimes I still let Gemini create my initial prompts or handle non-coding tasks, though.
But once I finish my setup completely, I could even run everything in my local environment.
And since I am currently developing my own personal model and framework, I am on my way to being completely independent from everything on the market and have the total control xD

But here is an overview for you in smaller pieces:

Overview:

First of all, you need to know which environment you are working in.
Mine is:

• Windows running WSL2 with Ubuntu 24.04, with 32 GB of DDR5 RAM allocated to the WSL distribution.
• GeForce RTX 4070 Ti with 12 GB of VRAM in total.
• VSCode with a AI coding extension like Continue. (You can find a lot of information and tutorials on integrating WSL in Windows and VSCode by googling it.)
  • Side note: I switched to the Zoo Code extension since Continue is no longer maintained.
• Ollama installed on the WSL distribution.

Most of my models are Qwen3. Additionally, I am currently trying Ornith, and it is really good at coding tasks (though sometimes it overthinks, xD).

ornith:latest                   a75697c14589    5.6 GB    4 days ago
qwen3:8b                        500a1f067a9f    5.2 GB    4 days ago
qwen3-coder-optimised:latest    791c8a24dedf    18 GB     4 days ago
qwen3-embedding:4b              df5bd2e3c74c    2.5 GB    4 days ago
qwen3-coder:30b                 06c1097efce0    18 GB     4 weeks ago

Important Information you have to check by yourself:

• Your GPU's VRAM.
• Your CPU cores (optional).

Creating custom model from a downloaded one:

You can see this list if you run ollama list inside the WSL command line after installing Ollama and pulling the models.
Basically, the size of the model is the most important metric for you. It tells you how much VRAM you need to run a model consistently.
Ollama will split the model internally if it sees that the GPU does not have enough VRAM.
However, Ollama's internal splitting can be inconsistent because it happens at runtime.
I prefer to have more control over this.

The next important piece of information is how many hidden layers the selected model has.
You can find this by searching for the model on Hugging Face and inspecting its config.json file.
Search for num_hidden_layers there.

Here an example from qwen3-coder:30b:

{
// ...
"norm_topk_prob": true,
"num_attention_heads": 32,
"num_experts": 128,
"num_experts_per_tok": 8,
"num_hidden_layers": 48, // <---- This is the important information
"num_key_value_heads": 4,
"output_router_logits": false,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
// ...
}

So now, it's just math.
Take the layer value (48). You can look at it like this: 48 layers = 18 GB model (this isn't the exact formula, but it achieves what we want).
Now you need to know how much VRAM your system uses while idle (check the Task Manager).
The rest is your available VRAM for the model.
Now, take 60%-70% of that available VRAM (we need space for the context window too, so we leave 30%-40% of the remaining VRAM for other tasks).

Now you need to calculate the layer amount that fits your GPU.
As an example using my specs:

• 12 GB VRAM total
• Windows uses ~3-4 GB
• 8 GB remaining
• 10% of 8 GB = 0.8 GB. 0.8 GB * 7 (for 70%) = 5.6 GB (I round it up to 6 GB).

Now we can calculate the layer amount for 6 GB, which is basically 1/3 of the total size in our calculation.
48 / 3 = 16 (I played around with it and figured out that 20 is a good sweet spot for me).
Now you have your num_gpu value for your custom version of the model.
To create a manually split model, you just need to execute ollama run <model-namefrom-list>.
Now you are in a CLI and can set the parameters.
First the num_gpu:
/set parameter num_gpu 20
(optional) I additionally configured the CPU threads that should be used:
/set parameter num_thread 12 (My CPU has 12 physical cores and a total of 24 logical cores. You will need to check your own CPU specs for this).

Now you can save it:
/save <custom-name>
And close ollama:
/bye

Now you have your custom model.

Those were the basic preparations.
Now, any API that calls Ollama will find your custom model, and you can use whichever APIs or frameworks you want.

Zoo Code

To answer your specific question: No, you do not need an MCP server just to let the agent read your project files. Extensions like Zoo Code or Continue can read your local workspace natively. An MCP server is only needed if you want to give the AI access to external tools or complex APIs.

For setting up Zoo Code, I highly recommend checking out their documentation. It is really well written and gives you everything you need to use the extension: https://docs.zoocode.dev/
Zoo Code already includes everything you need to work with your environment (reading and writing files, base-orchestral modes, in-depth configuration, skills, rules, and its own MCP marketplace). Best of all, it is community-driven and open-source.

With a free account on Qdrant, you can even add codebase indexing.
But as I said, since it has a lot of settings, it is better to consult the docs.
And basically, it is not an AI framework like Odysseus; it is an AI coding extension.

[SirThinksAl0t]

The crashes happened to me too.
I'm currently using LM Studio to download and setup models. So far no problems with it.

[bitboody]

The Cookbook feature is still under development and currently isn't as reliable as should be on Windows. For the time being, try using Ollama to download local LLMs.