Skip to content

Releases: sawhney17/droid-quality-audit

DROID-100 Quality Audit v0.1.0

Choose a tag to compare

@sawhney17 sawhney17 released this 10 Aug 02:00

DROID-100 quality audit v0.1.0

This release publishes a reproducible correction for the official droid_100/1.0.0 action description and a media-free quality-audit sidecar for all 100 episodes.

Deterministic finding

  • Every decoded action vector—32,212/32,212—exactly equals action_dict.cartesian_position (6 values) concatenated with action_dict.gripper_position (1 value); maximum absolute difference is 0.
  • The published features.json instead describes the first six values as joint velocities. The release includes an RFC 6902 patch and the full all-step audit.
  • Under the explicitly declared path-token mapping, terminal rewards match on 100/100 episodes; this is a consistency check between related source fields, not independent relabeling.

Provisional semantic screening

  • 100 consensus records, each backed by at least two distinct accepted model audits.
  • Gemini and Terra cover 100/100 episodes; Claude covers 99/100. The one missing model/episode pair is disclosed rather than coerced.
  • Model-derived episode labels remain candidates requiring human review. The release includes 533 typed attempt events and 52 sanitized terminal-failure invocation records, but no source media or provider response bodies.

Integrity

  • Full pinned-source, telemetry, decoded-frame, raw-response replay, consensus-reconstruction, supplemental-value leak, and release-checksum gates passed.
  • 170 offline tests, Ruff, strict mypy, JavaScript syntax, wheel build, and clean-wheel installation passed.

Read the report, methodology, and dataset card. Verify the downloadable bundle with release-assets-SHA256SUMS; its contents also carry an internal SHA256SUMS manifest.