Experiment results: small open LLM for daily Python (29–30 Aug 2026)

[YauhenBichel]

I tried a small open LLM for daily Python: ask, write a test, fix a bug, add one function. One laptop. 29–30 August 2026.

Not everyday-ready. Everyday-ready still means beating an untuned llama3.1:8b on live parse and a real ≥1 KB fix.

The tables with the planted example live on Pages: Experiments. What you type: Scenarios. Same numbers in the README.

Results

Experiment	Example	Result
0.5B as daily work	weekday helper, count-md, Action:	0 / 4 vibe, 0 / 2 parse
Four Start commands	demo/orders, subtotl / stauts	0 / 4, then 4 / 4 after the harness
Which open model	same bench, code must run	8B 9 / 9; 7B coder 7 / 9; 30B timeout
Train more?	35 pairs, 30 traces	No. Later ~2k clean turns
Larger model on a GPU	--engine openai	No live 14B / 32B number yet

Four commands, as typed

I typed	First night	After the harness
ask "what does compute_total return?"	"int"	Type plus what it computes
run "write tests for apply_discount"	Dead test below if __name__	Already covered. Nothing written
run "find the NameError and fix it"	Three files edited	subtotl → subtotal. No model
run "add a function total_lines and a test"	Opened a file. Suite red	total_lines(prices) + test. No model

Live first-Action parse (eval_everyday.py --live, llama3.1:8b): 8 / 15.

A leftover NameError (stauts inside def status) asks what to return. It does not invent return "ok". Answering ok writes that literal with no model.

What I am not doing

• More 0.5B train steps. The adapter overfit after step 100.
• An 8B LoRA on thirty seed traces.
• Calling the project everyday-ready from one parse pass.

This thread is for questions about the numbers. Design that is not a measurement stays in the other research discussions (eval protocol, fused weights).