v4.0.0-beta: Ollama + Project Reorganization + Smart Model Saving

What's new

Project Reorganization

• Gen2 code promoted to repository root — no more gen2/ prefix
• Gen1 archived to old/ (untracked)
• Security-hardened .gitignore — secrets, model weights, logs, and debug artifacts are all excluded

Ollama Support (Local LLM)

AI Supervisor now supports Ollama as a provider — run hyperparameter tuning with local models, no API key needed:

python trainer.py --resume --ai-supervisor ollama

• Configurable via OLLAMA_HOST and OLLAMA_MODEL env vars
• Default model: llama3.1
• Uses stdlib urllib — no extra dependencies

Smart Best Model Saving

Best model checkpoints are now saved based on fitness (survival + food) instead of raw reward:

fitness = avg_steps + avg_food × 10

• File naming: best_model_{uid}_ep{ep}_s{steps}_f{food}.pth
• Steps (survival time) dominate the score, food amplifies it
• Prevents saving "lucky" models that got high reward from a single good episode

Stage 1 Extended Episodes

• max_steps increased from 300 → 600 (~48 sec instead of ~24 sec)
• Gives the snake more time to learn food-seeking behavior before episode timeout

AI Supervisor in Progress Report

• New "AI Supervisor — Recent Changes" table at the bottom of progress_report.md
• Shows last 10 consultations: time, episode, parameter changes, reasoning
• Tracks most frequently adjusted parameters

TUI Dashboard

• Added screenshot of the training TUI to README

Providers supported

Provider	Flag	API Key Required
Claude	--ai-supervisor claude	Yes (ANTHROPIC_API_KEY)
OpenAI	--ai-supervisor openai	Yes (OPENAI_API_KEY)
Gemini	--ai-supervisor gemini	Yes (GOOGLE_API_KEY)
Ollama	--ai-supervisor ollama	No

Full changelog

• Reorganize: gen2/ → root, gen1/ → old/
• .gitignore: block .env, *.pth, logs/, config_ai.json, .claude/, chart_*.png
• ai_supervisor.py: add _call_ollama() via urllib.request
• trainer.py: fitness-based best model saving, ollama in --ai-supervisor choices
• styles.py: S1 max_steps 300 → 600
• training_progress_analyzer.py: parse ai_supervisor.log, render changes table in MD report
• auto_report.sh: fix paths after reorganization (remove gen2/ prefix)
• README.md: add TUI screenshot, update AI supervisor docs for Ollama

v3.0.0-beta: Curriculum Enemy Avoidance

What's new

Curriculum S3: Enemy Avoidance

The bot now progresses through a 4-stage curriculum where Stage 3 specifically targets enemy snake avoidance with tuned reward weights:

• Strong enemy proximity penalty (1.5) with large detection radius (1500 units)
• Enemy approach penalty (0.5) teaches the bot to turn away from approaching snakes
• Harsh snake death penalty (-40) makes dying to enemies costly
• Food reward kept high (5.0) so the bot doesn't forget how to eat

Training Analyzer Fixes

• Fixed death cause parsing — the analyzer was capturing "Q" instead of the actual death cause after Q-value columns were added
• New chart: MaxSteps Analysis (chart_16) — shows steps per episode vs the max_steps limit per curriculum stage, plus hit rate breakdown

Documentation

• Complete rewrite of README with human-readable explanations of:
  • How the neural network processes game state (CNN + sector vector)
  • Full reward system breakdown with per-step and death penalty tables
  • Curriculum stage progression with reward weight comparison table
  • Training infrastructure (browser engine, multi-agent, PER)

Tuning

• S2 promote threshold lowered (80→55 steps) to prevent bots from getting stuck when they already mastered wall avoidance
• S3 max_steps increased to 1000 for longer practice episodes

Training Status

• Bot successfully progressing through S3 with SnakeCollision deaths dropping from 100% to ~77%
• Episodes lengthening (avg 83 steps vs 73 before S3)
• Q-values stable and rising (mean 18-27)

Alpha-3: 4-Stage Curriculum + N-Step DQN + Pro Analytics

Alpha-3 — Full Training System Overhaul

4-Stage Curriculum with Dynamic Gamma

Stage	Name	Gamma	Promote Criteria
S1	FOOD_VECTOR	0.80	avg_food>=5 AND avg_steps>=50 (200 ep window)
S2	WALL_AVOID	0.90	avg_steps>=80 AND wall_death<3% (200 ep window)
S3	ENEMY_AVOID	0.95	avg_steps>=120 (200 ep window)
S4	MASS_MANAGEMENT	0.99	Final stage

Core Changes

• N-step returns (n=3) — reward signal propagates 3x faster
• Dynamic gamma per stage (effective horizon: 5→10→20→100 steps)
• Compound promotion — S1 requires BOTH food AND survival thresholds
• Tightened criteria — prevents premature stage advancement (previous run stuck at S3 after 8k eps)
• New rewards: enemy_approach_penalty, boost_penalty, expanded food_shaping clamp
• Softened S3 enemy penalties (2.0→1.0 proximity, 1.0→0.5 approach) to prevent paralysis

Professional Analytics Suite (13 Charts)

1. Main Dashboard (Bollinger bands, percentile steps, food SMA, loss, epsilon+LR, death area)
2. Curriculum Stage Progression (timeline with color bands)
3. Per-Stage Distributions (violin + box plots)
4. Hyperparameter Tracking (epsilon, LR, beta, loss EMA)
5. Correlation Scatter Matrix (12 plots with Pearson r + R² trend lines)
   5b. Correlation Heatmap + Ranked Strongest Correlations
6. Rolling Performance Percentile Bands (P10-P90)
7. Death Analysis (pie, grouped bars, histogram, rolling heatmap)
8. Food Efficiency (ratio tracking, cumulative, per-stage box)
9. Reward Distributions per Stage
10. Learning Detection (CDF early/late, volatility, segmented trends)
11. Goal Progress Gauges
12. HP Analysis (parallel coordinates, Epsilon×LR sweet spot heatmap, top 5% vs bottom 5%)

Breaking Changes

• Requires full training reset (new checkpoint format with n-step buffers + dynamic gamma)
• Old 3-stage curriculum replaced with 4-stage
• Old chart PNG files removed from repo

Previous Issues Fixed

• Bot stuck at Stage 3 after 8000+ episodes (avg_steps=42, 99.7% snake death, flat rewards)
• Root cause: S1 promoted after only 100 eps with trivial avg_food>=3 threshold
• S2→S3 promoted without real wall avoidance mastery

Alpha-2

v1.1.78

stats

alpha-1

v1.1.0

auto: update training reports 2026-02-05 10:14

Alpha

Alpha