Get parameters and results of all the trials

[riyadparvez]

AxClient class has get_best_parameters() method. I don't see any methods in AxClient class that returns all the trials with their parameters and evaluation results. Is there any plans to add such API?

[dongyaoli]

AxClient class has get_best_parameters() method. I don't see any methods in AxClient class that returns all the trials with their parameters and evaluation results. Is there any plans to add such API?

I think you can get the full results using the experiment object inside the client

ax = AxClient()
result = ax.experiment.fetch_data()

[riyadparvez]

This is what I see if I do print(result.df)

  arm_name metric_name      mean  sem  trial_index
0      0_0   hartmann6 -0.016126  0.0            0
1      0_0      l2norm  1.384382  0.0            0
2      1_0   hartmann6 -0.114288  0.0            1
3      1_0      l2norm  0.663507  0.0            1
4      2_0   hartmann6 -0.009832  0.0            2
5      2_0      l2norm  1.874537  0.0            2

Still no easy way to get the parameters and the metrics.

[lena-kashtelyan]

@riyadparvez, in the short run you can get trials as ax_client.experiment.trials, and you can get parameters from each one of the trials as trial.arm.parameters. Will add this functionality to the service API, too, but that should get you unblocked for now.

[riyadparvez]

@lena-kashtelyan thanks, it worked.

Is it also possible to expose which trial yielded the best results?

[lena-kashtelyan]

Hello again, @riyadparvez –– currently get_best_parameters outputs a tuple of parameterization and metric values; you can use the former as a workaround for now, to find which trial output the results.

get_best_parameters is still somewhat under development; thank you for the good idea of adding the trial index to its output!

[lena-kashtelyan]

This is now on master! You can do ax_client.get_trials_data_frame() : )

[lena-kashtelyan]

This functionality is now available in the latest stable version, 0.1.6!

[riyadparvez]

Thanks a lot! Will give it a try.