dev agents agent_encode

Agent Encode Open Research

it's easy to encode something somehow, but we need strict evals to validate agent understanidng

• How can we encode flat map for an agent to understand?

  • someone worked on this for maps (without success but good convo to understand our eval considerations https://github.com/dimensionalOS/dimos/pull/822)
  • how many rooms there are?
  • where is the smallest room?
  • take a hallway then turn left
  • go to second room on your right
  • approaches for this: VLM understanding costmap image seems difficult (see the PR above) but maybe numpy array agent can investigate with tools? some goofy ascii encoding? json? idk

• How can we encode a pointcloud for an agent to understand?

  • egocentric probably very important.
  • how good is agent at using py repl to investigate a PointCloud2?