Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added MLflow experiment tracking support #1191

Merged
merged 14 commits into from
Jun 3, 2021
Merged

Added MLflow experiment tracking support #1191

merged 14 commits into from
Jun 3, 2021

Conversation

tgaddair
Copy link
Collaborator

@tgaddair tgaddair commented Jun 3, 2021

This PR adds support for MLflow as a contrib package used to log experiment results during training and hyperopt.

MLflow can be enabled programmatically via callback:

model = LudwigModel(..., callbacks=[MlflowCallback()])

Or on the command line:

ludwig train ... --mlflow

When logging experiment results, Ludwig's experiment_name will serve as the Experiment used to organize results in MLflow. When using hyperopt features such as Ray Tune, every trial will be recorded as a separate run within the experiment.

Hyperopt and training parameters are logged as params, and all progress metrics are reported as metrics per epoch.

Additionally, the user's config file, description.json, training_progress.json, and model folder are all uploaded as artifacts.

@tgaddair tgaddair requested a review from w4nderlust June 3, 2021 01:44
@tgaddair tgaddair merged commit fbb0b58 into master Jun 3, 2021
@tgaddair tgaddair deleted the mlflow branch June 3, 2021 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants