Evaluation point for PointMass environment

[tldoan]

Hi,

I understand that for evaluation phase for the AntMaze you just let the Ant start from init_pos
https://github.com/florensacc/rllab-curriculum/blob/master/curriculum/experiments/starts/maze/maze_ant/maze_ant_brownian_algo.py#L140-L155

But for PointMass is it possible to have a list of init_pos too?
I tried to understand your code :
apparently you are using test_policy methods

rllab-curriculum/curriculum/experiments/starts/maze/maze_brownian_algo.py

Line 157 in a1eb78b

test_and_plot_policy(policy, env, as_goals=False, max_reward=v['max_reward'], sampling_res=sampling_res,

But I couldn t find the method : update_init_selector that apparently modifies the initial states when resetting the env?

rllab-curriculum/curriculum/envs/maze/maze_evaluate.py

Line 219 in 81a3714

train_env.update_init_selector(FixedStateGenerator(init_state))

https://github.com/florensacc/rllab-curriculum/search?q=update_init_selector&unscoped_q=update_init_selector

Thank you very much.

[florensacc]

Hi tlss94,

You are right, the function "update_init_selector" shouldn't be used there. This bug does not affect the default execution of the evaluation: when calling test_and_plot_policy, this calls test_policy, which has as default argument parallel=True, therefore returning with test_policy_parallel instead of executing any of the code you were pointing at. You can see that the parallel evaluation uses evaluate_states: https://github.com/florensacc/rllab-curriculum/blob/master/curriculum/envs/maze/maze_evaluate.py#L312

and this in turn ends up using env.update_start_generator(FixedStateGenerator(state)), which is valid. https://github.com/florensacc/rllab-curriculum/blob/master/curriculum/state/evaluator.py#L274

To reply to your question, you can see in the first link I paste here that the states that are evaluated come from the functions tile_space or find_empty_spaces. Is this what you were looking for? If you wish to have some other custom set of stats tested you can modify that part of the code.