You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Show & tell: tinyCode, keeping the context small so local models finish at 16k
#12146
Hi all. I want to share a small pet project I have been working on.
Motivation: I have been testing local models on my own device. On a 16 GB GPU a good coding model fits, but often with only ~16k of context. Most harnesses I tried struggled with that; Pi worked, but its loop fills the context fast.
Solution: tinyCode is a Pi package that does the context management for the model. It keeps only what the current step needs, finds the right files and the failing test itself, and asks the model for one small edit instead of letting it explore. Three things you notice: it rarely runs out of room, it doesn't say "fixed" unless a test that was failing now passes, and if a run goes wrong you keep the best version it reached. Everything is behind a switch, and pi --no-extensions gives you plain Pi to compare.
In more detail, what changes when you use it:
It doesn't run out of room. Each call is put together for the step at hand and old tool output is dropped, so a 16k window holds up through a long run.
It doesn't slow down as the run gets longer. The start of the prompt stays the same between calls, so your server reuses what it has already read.
You don't have to pick a mode. Say what you want in plain words; it chooses the workflow and tries the cheapest fix first.
Its edits apply. An edit that is slightly off still lands, instead of failing and costing another round.
You don't lose good work. If a run gets worse towards the end, you keep the best version it reached.
"Fixed" means fixed. It runs the tests itself, in Python and TypeScript/JavaScript, and only reports a fix when a failing test now passes.
It doesn't go round in circles. When a run stops improving, it changes approach instead of repeating itself.
Plans start from the code. /plan reads the files it will change before the plan is final, so plans are less likely to miss the file that needs the edit.
Results: On Aider's connect exercise with Qwen3.6-35B-A3B, plain Pi rewrote the file twice and was cut off at 8/10 tests; with tinyCode it made one targeted edit and passed 10/10 in 30 s.
Where it helps and where it doesn't:
It helps most when the window is the limit: a 16k window, a mid-size model (I tested gpt-oss-20b and Qwen3.6-35B-A3B), and a task with a failing test to fix. On five public exercises it took gpt-oss-20b from 3/15 to 8/15 and Qwen from 4/15 to 10/15.
On tasks that fit easily in the window, plain Pi is faster. tinyCode adds steps that only pay off when room is short.
Small models on 8 GB cards don't benefit yet.
Runs vary: the same task can pass in 30 s on one run and hit the 10-minute cap on the next.
/build finishes only the first task of a plan on its own; the model does the rest by hand.
It sometimes picks the wrong workflow, for example for a rename or a bug described only by its symptom.
The numbers above come from a small set. A 49-task comparison against plain Pi runs tonight, and I'll post the results here.
Every feature is measured against plain Pi first; this release removed one that didn't help.
If you run local models with a small window, please try it on a real task in your own project and reply here with your card, model and window, and what happened, good or bad. Those reports decide what I work on next. If something breaks, an issue on the repo with the task and the log is the most useful thing you can send.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi all. I want to share a small pet project I have been working on.
Motivation: I have been testing local models on my own device. On a 16 GB GPU a good coding model fits, but often with only ~16k of context. Most harnesses I tried struggled with that; Pi worked, but its loop fills the context fast.
Solution: tinyCode is a Pi package that does the context management for the model. It keeps only what the current step needs, finds the right files and the failing test itself, and asks the model for one small edit instead of letting it explore. Three things you notice: it rarely runs out of room, it doesn't say "fixed" unless a test that was failing now passes, and if a run goes wrong you keep the best version it reached. Everything is behind a switch, and
pi --no-extensionsgives you plain Pi to compare.In more detail, what changes when you use it:
/planreads the files it will change before the plan is final, so plans are less likely to miss the file that needs the edit.Results: On Aider's
connectexercise with Qwen3.6-35B-A3B, plain Pi rewrote the file twice and was cut off at 8/10 tests; with tinyCode it made one targeted edit and passed 10/10 in 30 s.Where it helps and where it doesn't:
/buildfinishes only the first task of a plan on its own; the model does the rest by hand.Source, guide and six side-by-side tasks
Every feature is measured against plain Pi first; this release removed one that didn't help.
If you run local models with a small window, please try it on a real task in your own project and reply here with your card, model and window, and what happened, good or bad. Those reports decide what I work on next. If something breaks, an issue on the repo with the task and the log is the most useful thing you can send.
All reactions