Improve html viz

[vcharraut]

Summary

This PR updates obs HTML generation.

Changes

• Refactored C observation writing into smaller pieces for ego, target, partner, road, and traffic control observations.
• Added compact obs HTML frame export path via vec_get_obs_html_frame, reducing Python-side replay state extraction overhead.
• Expanded interactive replay HTML:
  • policy outputs, values, entropy, action probabilities/densities
  • pool slot counts
  • steering, accel, jerk, lane id
  • Puffer score components
  • improved playback timing/speed controls
• Added evaluator progress bars for triage/obs HTML generation.
• Updated Drive backbone observation slicing and added pool_slot_counts() for visualization/debugging.
• Added shared notebook utilities and refreshed notebooks.

[Copilot]

Pull request overview

This PR refactors PufferDrive’s observation-generation pipeline and associated configuration surface, while extending/improving the HTML/visualization tooling for debugging and triage.

Changes:

• Renames/reshapes many env config keys (obs slot counts, dropouts, normalization scales, and range parameters; init_steps → init_step; reward rename).
• Refactors C-side observation construction into helper routines and updates Python/C bindings + Torch encoder to the new slot/dropout semantics.
• Enhances obs-html rendering by collecting compact per-step arrays (agents/metrics/traffic/policy outputs) and adds a C binding (vec_get_obs_html_frame) to fill them efficiently.

Reviewed changes

Copilot reviewed 36 out of 38 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
weights/tomate/config.yaml	Updates env config keys to new obs_* naming scheme and reward/init renames.
weights/salade/config.yaml	Same config key migration as tomate.
weights/oignons/config.yaml	Adds new experiment config using new obs_* keys.
weights/oignons2/config.yaml	Adds new experiment config using new obs_* keys (left-map variant).
tests/test_eval_manager.py	Updates tests to use renamed dropout key.
tests/test_drive_config.py	Updates tests to use renamed reward keys.
scripts/render_scenario.py	Renames CLI flag and env override keys (init_step, obs_dropout_*).
README.md	Updates documentation to the renamed collision reward key.
pufferlib/utils.py	Updates render CLI invocation to --init-step.
pufferlib/pufferl.py	Updates env-yaml key allowlists to new observation key names.
pufferlib/ocean/torch.py	Updates policy encoder slicing/shape logic to new slot/dropout/kept counts.
pufferlib/ocean/env_config.h	Renames env config fields + INI handler keys for new obs/reward/init naming.
pufferlib/ocean/env_binding.h	Adds vec_get_obs_html_frame binding and consolidates ego feature constant export.
pufferlib/ocean/drive/visualize.c	Updates visualize CLI flag parsing and Drive struct field names for init/control counts and rewards.
pufferlib/ocean/drive/render.h	Updates observation visualization scaling and slot iteration to new obs normalization/range fields.
pufferlib/ocean/drive/README.md	Updates init-mode documentation to init_step.
pufferlib/ocean/drive/drivenet.h	Aligns ego feature sizing with unified EGO_FEATURES.
pufferlib/ocean/drive/drive.py	Updates Python Drive env to new config keys and adds get_obs_html_frame API.
pufferlib/ocean/drive/drive.h	Refactors observation writing into helper functions; renames many Drive fields; adds projection helpers.
pufferlib/ocean/drive/drive.c	Updates demo/perf setup to new fields and fixes forward() call type.
pufferlib/ocean/drive/datatypes.h	Adds explicit traffic-control scope constants.
pufferlib/ocean/drive/binding.c	Extends exported agent state/metrics for viz and updates init kwargs unpacking to new keys.
pufferlib/ocean/benchmark/visual_sanity_check.py	Updates WOSAC setup to use env.init_step.
pufferlib/ocean/benchmark/metrics_sanity_check.py	Updates WOSAC setup to use env.init_step.
pufferlib/ocean/benchmark/manager.py	Updates clean-macro overrides to renamed dropout keys.
pufferlib/ocean/benchmark/evaluators/base.py	Adds tqdm progress and rewrites obs-html capture to compact-array schema via new binding.
pufferlib/ocean/benchmark/evaluator.py	Switches WOSAC init-step sourcing to env.init_step.
pufferlib/config/ocean/drive.ini	Renames keys (init_step, reward_*, obs_*) and reorganizes observation-related settings.
notebooks/notebook_utils.py	Adds shared notebook helpers/constants for env/policy setup and dimension derivation.
notebooks/06_architecture.ipynb	Refactors notebook to use notebook_utils helpers and new obs-slot keys.
notebooks/05_inference.ipynb	Refactors notebook setup and updates dimensions/labels for unified ego features + new keys.
notebooks/04_training.ipynb	Refactors notebook to use notebook_utils helpers and new obs-slot keys.
notebooks/03_metrics.ipynb	Refactors notebook to use notebook_utils helpers and new obs-slot keys.
notebooks/02_rewards.ipynb	Refactors notebook to use notebook_utils helpers and fixes goal extraction indices under new layout.
notebooks/01_observations.ipynb	Refactors notebook to use notebook_utils helpers and new obs-slot keys.
notebooks/init.py	Marks notebooks as a package for importing notebook_utils.
docs/evaluation.md	Updates clean-macro docs to the renamed dropout keys.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[eugenevinitsky]

don't want to block, is non-breaking