Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor run_ml() API for future expandability #251

Closed
kelly-sovacool opened this issue Apr 29, 2021 · 1 comment
Closed

Refactor run_ml() API for future expandability #251

kelly-sovacool opened this issue Apr 29, 2021 · 1 comment
Assignees
Labels
JOSS-paper wontfix This will not be worked on

Comments

@kelly-sovacool
Copy link
Member

From openjournals/joss-reviews#3073 (comment):

The run_ml() function currently implements 5 off-the-shelve ML algorithms, while providing 12 other parameters for training criteria, hyperparameters, feature importance, etc.. If in the future it would support more algorithms, custom metrics, or training parameters, I'd imagine there'll be limitations imposed by the function arguments. I'd suggest the function to take in 3 objects, e.g. run_ml(dataset, model, metrics, [args]), where a metrics object can allow the user select standard metrics or define their own metric functions given the model output and true labels.

@zenalapp
Copy link
Collaborator

run_ml() already takes these objects:

  1. dataset: The input dataset.
  2. method: The ML model to be used. While we only officially support 5 models, all of the models supported by caret (https://topepo.github.io/caret/available-models.html) should work in our package. If caret supports additional models in the future, these should also work in mikropml. We realize that the model options are not as generalizable as e.g. PyTorch, since users must choose from options supported by caret. However, our code heavily relies on caret to perform the underlying model training. Additionally, as mikropml is oriented toward beginner practitioners, we believe that it does not need to provide the option to include custom models.
  3. perf_metric_function and perf_metric_name: The performance metric to be used. We chose sensible defaults, but the user can provide their own performance metrics if they would like.
  4. hyperparameters: The values of hyperparameters in the model that the user would like to tune.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
JOSS-paper wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

2 participants