dev agents agent_encode

Jump to bottom

leshy edited this page Mar 24, 2026 · 9 revisions

Agent Encode Open Research

How can we encode flat map for an agent to understand?
- someone worked on this for maps (without success but good convo to understand our eval considerations https://github.com/dimensionalOS/dimos/pull/822)
- how many rooms there are?
- where is the smallest room?
- take a hallway then turn left
- go to second room on your right
- approaches for this: VLM understanding costmap image seems difficult (see the PR above) but maybe numpy array agent can investigate with tools? some goofy ascii encoding? json? idk
How can we encode a pointcloud for an agent to understand?
- egocentric probably very important.
- how good is agent at using py repl to investigate a PointCloud2?