Best practices for local Ollama (RTX 3090) + Cline: Handling timeouts, token limits, and improving Autonomy

[tomasenskede]

Hi everyone,

I'm currently building custom agents and integrations for my e-commerce platform using Cline, connected to a local Ollama instance. My hardware is a single RTX 3090 (24GB VRAM). To optimize performance and eliminate VRAM thrashing, I recently locked my environment to a "Single-Model Policy" using qwen3.8:27b.

While this made the baseline highly stable, I am struggling with two main friction points when running more complex workflows. I would love some advice from anyone running a similar local setup:

1. Heavy generations trigger timeouts and token limits
   When Cline attempts to write or append large chunks of code (for example, inserting Phase 1 types and multiple functions into a 400+ line TypeScript file), the local model struggles to output everything in one go. I frequently hit these three errors:

- terminated: BodyTimeoutError: Body Timeout Error (UND_ERR_BODY_TIMEOUT)

• Model reached the maximum output token limit before completing the turn (generation cuts off mid-code block).
• "Empty client" responses (the model seemingly returns nothing or drops the connection silently).

Cline usually tries to recover (e.g., running wc -l to find the end of the file and try again), but it often breaks the flow completely.

• Question: Is there a way to explicitly increase the HTTP request timeout and max output tokens for the Ollama provider in Cline? Furthermore, is there a recommended custom system prompt to force Cline to chunk its file edits into smaller, more digestible operations?

2. True Autonomy for long-running tasks
   I want to be able to start a larger refactoring or testing task late at night and let Cline run autonomously. However, it often halts after a short while, waiting for manual user approval to run a terminal command or edit a specific file.

• Question: How do you configure your "Auto-approve" settings for safe, unattended operations? Are there specific boundaries you set so it doesn't get stuck in a loop or do destructive edits while you are away from the keyboard?

I'm looking to optimize this setup for a smoother, more deterministic developer experience. Any tips, configuration tweaks, or workflow advice would be highly appreciated!

Thanks in advance.

[159753a52]

Can't help much on the auto-approve side, but on a single 3090 the timeouts and empty replies are often a memory/context issue on the Ollama side, so here's the budget for qwen3.8:27b (estimates, not measured on your box):

• Weights: the qwen3.8:27b tag is 18 GB (~16.8 GiB, vision tower included). That leaves ~7 GiB of the 24, minus ~1-1.5 GiB for the CUDA context and compute buffers.
• KV cache: Qwen3.8 27B only keeps a KV cache in 16 of its 64 layers (the other 48 are linear attention with a small fixed state). 16 layers x 4 KV heads x 256 head_dim x 2 (K+V) x 2 bytes = 64 KiB per token at f16. So 32K ctx = 2 GiB, 64K = 4 GiB, 128K = 8 GiB, 256K = 16 GiB.

So ~64K fits comfortably at f16 and ~90K is about the ceiling. If the context is set to 128K+, Ollama has to put part of the model on the CPU and generation drops to a few tok/s, which is exactly when long Cline turns hit the body timeout or come back empty. ollama ps shows the context it actually allocated and the GPU/CPU split under PROCESSOR; you want 100% GPU. With OLLAMA_FLASH_ATTENTION=1 and OLLAMA_KV_CACHE_TYPE=q8_0 the cache roughly halves (~34 KiB/token), so 128K is ~4.25 GiB and still fits.

The other direction bites too: Cline's system prompt + a 400-line file + tool history grows fast, and when the prompt is longer than the context, Ollama truncates it rather than erroring, which can give those odd empty or confused turns. Ollama's own docs suggest at least 64K for coding agents.

For scale on speed: fully on the GPU, a 27B at ~Q4 on a 3090 should do roughly 26-36 tok/s at 32K context (936 GB/s / ~19 GB read per token ≈ 49 tok/s ceiling; real engines get 55-75% of that). A ~5K-token file rewrite plus thinking is a few minutes of generation, so nudging Cline toward replace_in_file edits instead of whole-file write_to_file helps with both the output-token limit and the timeouts.

[tomasenskede]

Thanks a lot for the detailed breakdown! Your math completely explains the exact symptoms I was seeing.

Limiting the context window and keeping everything within the 24GB VRAM budget (along with sorting out the template issues) fixed the timeouts and empty replies entirely. Your tip about steering the coding agent toward replace_in_file edits instead of whole-file rewrites also makes total sense to keep generation times manageable. Really appreciate the great explanation!

[tomasenskede]

Update: Significantly mitigated UND_ERR_BODY_TIMEOUT via strict rulesets (but data-heavy context still breaks it)

After extensive testing with a single-model approach (qwen3.8:27b quantized to Q4_K_M) on a 24GB RTX 3090, I wanted to share what works and where the hard limits still lie regarding the UND_ERR_BODY_TIMEOUT.

1. The Fix for Code/File Generation (The Snippet Strategy)
   When asking the agent to edit or create large files, it would constantly time out. I managed to stabilize this by injecting a strict Anti-Timeout Ruleset into my .clinerules / root context files.

Here is the exact markdown block I enforce:

## ⚠️ CRITICAL AGENT STABILITY RULES (ANTI-TIMEOUT)
To prevent UND_ERR_BODY_TIMEOUT and context window exhaustion when running on local models, you MUST strictly adhere to the following:

1. **Tool Constraints:** 
   - NEVER use `run_commands` or the terminal to read, search, or inspect files (e.g., do not use `cat`, `grep`, `less`).
   - ALWAYS default to using `read_files`, `search_codebase`, and `editor`.
   - Terminal is strictly reserved for package installation (e.g., `npm install`) or running explicitly requested scripts/migrations.
2. **Context Management:** 
   - NEVER search or read inside `.venv`, `node_modules`, `.git`, or log directories.
3. **Handling Large Files (The Snippet Strategy):** 
   - When dealing with massive files, DO NOT attempt large edits directly with the `editor` tool if the context is large. If a complex injection is needed, create a separate snippet file first (e.g., `snippet.js`) and wait for user confirmation.
4. **Token Economy:** 
   - Be extremely concise in your text responses. Skip pleasantries. Go directly to tool calls to conserve output tokens.
5. **Step-by-Step Execution:** 
   - Never attempt to solve a multi-step architectural change in one single output. Propose one tool call/action at a time.

Why the Snippet Strategy works: By forcing the agent to create a snippet.js file first, it avoids loading massive files into the editor tool memory. In one recent task at ~49k tokens, the agent realized its generated code exceeded the tool limits, so it autonomously split the creation of the snippet into two separate tool calls (writing lines 1-83 first, then appending the rest). No timeout!

2. The Catch: Raw Data Overload
While the rules above fixed the code-generation timeouts, I just hit a hard UND_ERR_BODY_TIMEOUT again. Why? Because the agent ingested a large Bank Statement CSV for reconciliation.

Once the context window spiked to 78,000 tokens (surpassing my num_ctx: 64000 limit), the agent successfully ran its Python scripts but completely choked when trying to write a simple text response file (REPLY_MAIL.txt). The sheer weight of the context history caused the timeout.

The takeaway for 24GB setups:
You can guide the agent around output timeouts using strict rules (like the snippet strategy), but if you flood the input context with massive raw data files (like large CSVs or logs), the 24GB VRAM ceiling will still enforce a brutal timeout when the agent tries to reason about it. Always start a "New Task" to clear the context after heavy data crunching before asking the agent to write out its conclusions.

I'd love to hear how others are navigating these limits! Have any of you found different workarounds for data-heavy tasks on 24GB setups, or are you also relying on frequent "New Task" resets to clear the context? Please share your experiences below!

[omgitsbase]

One more lever for the timeouts that are left, since they hit on long generations: raw decode speed. qwen3.8:27b has an MTP head, and how much you get out of it depends a lot on the runtime.

I build llmash, an Ollama-compatible server on a llama.cpp fork. Same API, and it reads your existing Ollama models, so Cline needs no changes. In my benchmarks the same 27B decodes at 162-173 tok/s on llmash against 110-148 for Ollama's MTP q4_K_M build. That was on an RTX PRO 6000 and I haven't measured a 3090, so treat the ratio as the rough expectation rather than the numbers. Disclosure: it's my project.