Skip to content

v0.2.0

Choose a tag to compare

@kiyeonjeon21 kiyeonjeon21 released this 16 Oct 17:53
· 25 commits to main since this release
d0fc08e

PhySim Gate is now on PyPI: pip install physim-gate.

Highlights

  • LeRobot adapter. --adapter lerobot reads the eval_info.json that lerobot-eval writes (LeRobot 0.4.0+ per_task layout and the 0.3.x per_episode layout). Evaluations run with the same --seed are compared episode by episode.
  • Real simulator example. examples/lerobot-pusht/ gates unedited lerobot-eval output from the PushT simulator (100 episodes per run): a rerun of the same policy passes; changing the action horizon fails on max_reward, while the success drop from 11% to 1% is honestly INCONCLUSIVE at 100 episodes.
  • physim-gate plan. How many trials does a verdict need? Exact verdict probabilities for 0/1 metrics using the engine's own score interval; a normal approximation for continuous metrics. Example: showing that a 90% success rate has not dropped by more than 5 points needs about 573 episodes per run.
  • Markdown summaries. --markdown FILE and --format markdown for pull-request comments and job summaries.
  • GitHub Action. uses: kiyeonjeon21/physim-gate@v0.2.0 runs the gate, writes all reports and the job summary, exposes verdict and exit-code, and can post or update a pull-request comment.

Fixes

  • Paths in the "wrote ..." line are no longer wrapped mid-name in narrow or redirected terminals.

Notes

  • The run-result and comparison schemas are unchanged (1.0); v0.1 results and configs work as before.
  • The pull-request comment option of the action has not yet been exercised on a real pull request.

Full changelog: CHANGELOG.md