Skip to content

[ENH] encoding minimization vs maximization #141

@fkiraly

Description

@fkiraly

I noticed an issue with the current API design - the sign (pun intended).

Namely, experiments can be minimization or maximization, e.g., sklearn experiments with the accuracy score, vs the MSE; or more classical optimization.

Currently, the API implicitly assumes minimization everywhere. But this becomes confusing ot the user as the SklearnCvExperiment expects metrics, so it raises the expectation that it handles the sign.

It is unclear where to handle this and is left to the user - there are two natural places of which one or both could be used:

  • the experiment - a tag lower_is_better: bool or direction: {"max", "min", "neither"} - or an __init__ arg
  • the optimizer, which could get an __init__ arg, or an arg to run

It seems more natural to have the sign "minmize" or "maximize" as part of the optimization problem, hence the experiment.

My favourite solution is the tag, from a user perspective, although I wonder what this would imply about the optimizer. I am confident that we could assume minimization and let the boilerplate handle the sign, although this has to be thought through.

FYI @SimonBlanke

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions