Skip to content

RoboCasa: subgoal-image conditioning (pi07) is LIBERO-only #384

@shuheng-liu

Description

@shuheng-liu

Problem

Eval-time subgoal-image conditioning (the subgoal{k} inputs consumed by pi07 low-level / pi07-paligemma policies) is wired only for LIBERO. make_subgoal_generator early-returns for anything else:

if not isinstance(cfg.env, LiberoEnv) or cfg.env.subgoal_source is None:
    return None

So evaluating a subgoal-conditioned policy on RoboCasa silently falls back to the policy's missing-key default (zero subgoals, mask off) — the subgoal channel is effectively dead for RoboCasa.

Why it matters

pi07 low-level / pi07-paligemma can't be evaluated faithfully on RoboCasa, since they're trained with subgoal conditioning and would see none at eval.

Suggested approach

  • Generalize the subgoal generator beyond LIBERO — either a RoboCasaLastFrameSubgoalGenerator, or make the generator env-agnostic over any subgoal_source v2.1 LeRobot dataset and add a subgoal_source field to RoboCasaEnv.
  • Update make_subgoal_generator to dispatch per env type.

References

  • src/opentau/scripts/eval.py:620 (make_subgoal_generator)
  • src/opentau/envs/subgoal.py (LiberoLastFrameSubgoalGenerator, SubgoalImageGenerator)

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions