Policy Inspector

Policy Inspector

rlens now shows what your network is doing, not just its loss curve.

The new policy inspector renders per-layer weight and gradient distributions over
training as a heatmap (x = step, y = value, colour = density) — watch weights spread or
saturate and gradients collapse, layer by layer. It's the introspection W&B/TensorBoard
make you wire up by hand, available with zero setup in the local dashboard.

What's new

• Policy inspector in the dashboard — per-layer weight/gradient distribution heatmaps,
  with a layer selector, placed under the reward chart.
• Distribution capture is Trainer-owned and uniform across PPO/DQN/SAC — no per-algorithm
  code. Weights and gradients are sampled (capped per tensor) and logged as histograms over
  training; gradient norms (grad_norm/* scalars) are still there too.
• --inspect-interval flag / inspect_interval_steps config knob to control snapshot
  cadence (0 = auto ≈ 50 snapshots/run, <0 = off).
• New telemetry endpoint /api/runs/{id}/histogram_series serving the full (downsampled)
  histogram-over-time series.

Compatibility

• No breaking changes. The inspector is on by default at a coarse cadence; pass
  --inspect-interval -1 to disable.

Install / upgrade: pip install -U rlens

Full changelog: v0.1.0...v0.2.0

rlens v0.1.0 — first release

First public release (alpha).

rlens is a local, zero-setup workbench for understanding, debugging, and comparing
reinforcement-learning runs on a single machine — PPO, DQN, and SAC on one shared trainer,
streamed live to a built-in observability dashboard. Full tour in the
README.

Install

pip install rlens
optional extras: rlens[box2d] (LunarLander) · rlens[atari] (image observations)

Python 3.11+; CPU or Apple Silicon (MPS), no CUDA required.

In this release

• PPO, DQN, SAC on a shared trainer and telemetry layer.
• Built-in dashboard: multi-run overlay, all-metrics grid, run-comparison table, config-diff,
  gradient norms, action histograms, and inline rollout video.
• Image observations & Atari (Nature-CNN encoder, frame-stacking, uint8 replay).
• Reproducible & resumable: per-run config/version/git snapshots, full-state checkpoints,
  best-policy saving, crash-safe --resume.
• Reproduced reference returns on CartPole, Acrobot, Pendulum, and LunarLander — both PPO and
  DQN solve LunarLander-v3. See benchmarks/.

Caveats

• Single-machine scope — not distributed / multi-GPU / hosted tracking.
• The Atari pipeline is verified end-to-end, but reproducing published Atari scores needs a GPU.
• Alpha: the API may change before 1.0.

MIT licensed.