[Question]: Hi, could you explain how system2 works on VLN-CE R2R?

[DarkNanGua]

Question

As far as I know, R2R requires outputting four discrete actions, so how is the point output by system2 mapped?

[yuqiang-yang]

We use oracle depth to convert the 2d pixels to 3d points in the world coordinate. The a shortest path follower (built in habitat) tracks this 3d points.

[kellyiss]

For Dual-System inference, the continous trajectory will be converted to discrete_actions.