v0.4.0 — Benchmark Against Stable-Baselines3

Overview

v0.4.0 introduces a benchmark comparison between the custom PPO implementation and Stable-Baseline3 on CartPole-v1.

This release validates the correctness and effectiveness of the custom implementation by comparing it to a widely used reference implementation.

Key Results

Custom PPO

• learns successfully and reaches high rewards (~150-250)
• shows correct learning dynamics
• exhibits some instability typical for simplified implementations

Stable-Baselines3 PPO

• achieves perfect performance (mean reward: 500)
• demonstrates fully stable policy learning

Insights

The comparison confirms that the custom PPO implementation is correct and capable of learning strong polices, while also highlighting differences in stability and optimization compared to a production-grade implementation.

Additions

• Stable-Baseline3 benchmark script
• evaluation summary output
• comparison-ready experiment structure

Next steps

• analyze performance gap in more detail
• improve stability of custom PPO
• extended benchmarking to more complex environments