Questions on Table 3 (AlfWorld)

[guosyjlu]

Hi,
Thanks for your great work! I have a question on Table 3, where results of Act and ReAct are reported as avg/best of 6. I am wondering where does 6 come from, given that the decoding strategy is greedy.
Thank you!

[ysymyth]

Hi @guosyjlu , as stated in the paper, "For robustness, we construct 6 prompts for each task type through each permutation of 2 annotated trajectories from the 3 we annotate." So 6 trials come from different selections of prompting examples!