You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I noticed that when the evaluation_function returns value of some NumPy type(I was able to reproduce with np.int64, np.int32, np.float32), optimize method crashes with the following exception:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/service/managed_loop.py", line 206, in optimize
loop.full_run()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/service/managed_loop.py", line 150, in full_run
self.run_trial()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/service/managed_loop.py", line 141, in run_trial
trial.fetch_data()
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/core/base_trial.py", line 259, in fetch_data
trial_index=self.index, metrics=metrics, **kwargs
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/core/simple_experiment.py", line 205, in _fetch_trial_data
return self.eval_trial(self.trials[trial_index])
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/core/simple_experiment.py", line 119, in eval_trial
not_none(trial.arm).parameters, None
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ax/core/simple_experiment.py", line 192, in evaluation_function_outer
"Evaluation function returned an invalid type. The function must "
Exception: Evaluation function returned an invalid type. The function must either return a dictionary of metric names to mean, sem tuples or a single mean, sem tuple, or a single mean.
@Irynei, this is intentional behavior –– the evaluation function is strictly typed, and it's return type is a union of a float, a tuple of two floats, and a dictionary of a string to such tuples, as documented in SimpleExperiment. We use python builtins for primitive types and convert from numpy primitives where necessary. However, you're right that for the evaluation function, we should allow numpy types –– will add that support shortly.
Is it much trouble to convert back from numpy to python primitive in the evaluation function, for now? You could use our utility ax.utils.common.typeutils.numpy_type_to_python_type, if you'd like.
@lena-kashtelyan, thanks for your explanation. I was expecting the evaluation function to work with numpy types, so for me, that was a surprising behavior.
No, It's not hard to convert them for now, but thanks for the suggestion.
I noticed that when the
evaluation_function
returns value of some NumPy type(I was able to reproduce withnp.int64
,np.int32
,np.float32
), optimize method crashes with the following exception:My environment:
Script to reproduce:
The text was updated successfully, but these errors were encountered: