Pytorch Autologging Description Added To Tracking.rst #3636
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
area/artifacts
Signed-off-by: karthik-77 <karthiks@ideas2it.com>
karthik-77
force-pushed
the
mlflow-pytorch-autolog-doc
branch
from
f0ec98f
to
e24d4d9
Compare
…dded Signed-off-by: karthik-77 <karthiks@ideas2it.com>
chauhang
reviewed
Nov 4, 2020
docs/source/tracking.rst
Outdated
|
|
||
| Call :py:func:`mlflow.pytorch.autolog` before your training code to enable automatic logging of metrics and parameters. See example usages with `Pytorch <https://github.com/chauhang/mlflow/tree/master/examples/pytorch/MNIST>`_. | ||
|
|
||
| In the current implementation, PyTorch autolog works with the `Lightning training loop <https://github.com/PyTorchLightning/pytorch-lightning/blob/master/pytorch_lightning/trainer/training_loop.py>`_. The respective metrics associated with ``EarlyStopping Callabacks`` and ``pytorch_lightning.trainer`` are automatically logged. It must be noted that in case of a multi-optimizer scenario (such as usage of autoencoder) by default only the parameters for the first optimizer would be logged. |
There was a problem hiding this comment.
@karthik-77 Please move the Optimizer note to the "Note" callout at the end (after the table)
There was a problem hiding this comment.
Yes i have moved the Optimizer note to the "Note" Callout at the end
chauhang
reviewed
Nov 4, 2020
docs/source/tracking.rst
Outdated
| | | For example, ``stopped_epoch``, ``restored_epoch``, | For example, ``min_delta``, ``patience``, ``baseline``, | | Best Pytorch model Checkpoint if training stops due to early stopping callback. | | ||
| | | ``restore_best_weight``, etc. | ``restore_best_weights``, etc | | | | ||
| +------------------------------------------+------------------------------------------------------------+-------------------------------------------------------------------------------------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| | ``pytorch-lightning trainer`` |Training loss;validation loss;average_test_accuracy; | ``fit()`` parameters; optimizer name; learning rate; epsilon | -- | Model summary on training start; `MLflow Model <https://mlflow.org/docs/latest/models.html>`_ (Pytorch model) on training end; | |
There was a problem hiding this comment.
@karthik-77 Please change the sequence of the table rows. Move the pytorch-lightning trainer row to be the 1st row in the table
There was a problem hiding this comment.
I have changed the order. First row is the pytorch lightning trainer and the callback comes next,
chauhang
suggested changes
Nov 4, 2020
Signed-off-by: karthik-77 <karthiks@ideas2it.com>
Signed-off-by: karthik-77 <karthiks@ideas2it.com>
smurching
reviewed
Nov 5, 2020
docs/source/tracking.rst
Outdated
| | | | | | | | ||
| | | | | | | | ||
| +------------------------------------------+------------------------------------------------------------+-------------------------------------------------------------------------------------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| |``trainer.callbacks.earlystopping`` |Training loss;validation loss;average_test_accuracy; | ``fit()`` parameters; optimizer name; learning rate; epsilon | -- | Model summary on training start; `MLflow Model <https://mlflow.org/docs/latest/models.html>`_ (Pytorch model) on training end; | |
There was a problem hiding this comment.
Should this be pytorch_lightning.callbacks.early_stopping? https://pytorch-lightning.readthedocs.io/en/stable/early_stopping.html#early-stopping-based-on-metric-using-the-earlystopping-callback
I don't think a trainer.callbacks.earlystopping module exists
There was a problem hiding this comment.
Yes. The change has been incorporated.
smurching
reviewed
Nov 5, 2020
docs/source/tracking.rst
Outdated
| +------------------------------------------+------------------------------------------------------------+-------------------------------------------------------------------------------------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| | Framework/module | Metrics | Parameters | Tags | Artifacts | | ||
| +------------------------------------------+------------------------------------------------------------+-------------------------------------------------------------------------------------+---------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| |``pytorch-lightning trainer`` | Training loss; validation loss;average_test_accuracy; | ``fit()`` parameters; optimizer name; learning rate; epsilon. | -- | Model summary on training start; `MLflow Model <https://mlflow.org/docs/latest/models.html>`_ (Pytorch model) on training end; | |
There was a problem hiding this comment.
Suggested change
| |``pytorch-lightning trainer`` | Training loss; validation loss;average_test_accuracy; | ``fit()`` parameters; optimizer name; learning rate; epsilon. | -- | Model summary on training start; `MLflow Model <https://mlflow.org/docs/latest/models.html>`_ (Pytorch model) on training end; | | |
| |Pytorch-Lightning Trainer`` | Training loss; validation loss;average_test_accuracy; | ``fit()`` parameters; optimizer name; learning rate; epsilon. | -- | Model summary on training start; `MLflow Model <https://mlflow.org/docs/latest/models.html>`_ (Pytorch model) on training end; | |
There was a problem hiding this comment.
The suggested change has been applied.
smurching
reviewed
Nov 5, 2020
There was a problem hiding this comment.
Pushed some edits & had a few suggestions, otherwise looks good thanks @karthik-77!
|
Hi Siddarth,
Thanks for pointing that out. I will work on the suggestion and update the
file.
Thanks and regards
karthik
…On Thu, Nov 5, 2020 at 10:39 AM Siddharth Murching ***@***.***> wrote:
***@***.**** commented on this pull request.
Pushed some edits & had a few suggestions, otherwise looks good thanks
@karthik-77 <https://github.com/karthik-77>!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#3636 (review)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ARRYHDFTJZQA34DS7WUHNFTSOIXSFANCNFSM4TJ4RXXQ>
.
|
|
@smurching request you to kindly sign off the commits. |
chauhang
approved these changes
Nov 6, 2020
Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>
…suggested changes Signed-off-by: karthik-77 <karthiks@ideas2it.com>
Signed-off-by: karthik-77 <karthiks@ideas2it.com>
Signed-off-by: Sid Murching <sid.murching@databricks.com>
Signed-off-by: Sid Murching <sid.murching@databricks.com>
harupy
force-pushed
the
mlflow-pytorch-autolog-doc
branch
from
5bdf0c9
to
f86db6d
Compare
eedeleon
pushed a commit
to eedeleon/mlflow
that referenced
this pull request
Nov 13, 2020
* Added the pytorch autolog descriptionin the tracking.rst file Signed-off-by: karthik-77 <karthiks@ideas2it.com> * The parameter description for the mutli optimizer scenario has been added Signed-off-by: karthik-77 <karthiks@ideas2it.com> * The review comment for the autolog docs has been addressed Signed-off-by: karthik-77 <karthiks@ideas2it.com> * The review comment for the autolog docs has been addressed Signed-off-by: karthik-77 <karthiks@ideas2it.com> * Update tracking.rst Signed-off-by: harupy <17039389+harupy@users.noreply.github.com> * updated the tracking.rst file according to the reviewers comment and suggested changes Signed-off-by: karthik-77 <karthiks@ideas2it.com> * Corrected a punctuation error in the tracking.rst file Signed-off-by: karthik-77 <karthiks@ideas2it.com> * Update docs, fix autologging docstring Signed-off-by: Sid Murching <sid.murching@databricks.com> * Docstring fixes and wording updates Signed-off-by: Sid Murching <sid.murching@databricks.com> * Fix lint Signed-off-by: Sid Murching <sid.murching@databricks.com> Co-authored-by: Siddharth Murching <smurching@gmail.com> Co-authored-by: Sid Murching <sid.murching@databricks.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: karthik-77 karthiks@ideas2it.com
What changes are proposed in this pull request?
Pytorch autolog description was added to the tracking.rst file.
How is this patch tested?
Documentation -NA
Release Notes
Is this a user-facing change?
(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)
What component(s), interfaces, languages, and integrations does this PR affect?
Components
area/artifacts: Artifact stores and artifact loggingarea/build: Build and test infrastructure for MLflowarea/docs: MLflow documentation pagesarea/examples: Example codearea/model-registry: Model Registry service, APIs, and the fluent client calls for Model Registryarea/models: MLmodel format, model serialization/deserialization, flavorsarea/projects: MLproject format, project running backendsarea/scoring: Local serving, model deployment tools, spark UDFsarea/server-infra: MLflow server, JavaScript dev serverarea/tracking: Tracking Service, tracking client APIs, autologgingInterface
area/uiux: Front-end, user experience, JavaScript, plottingarea/docker: Docker use across MLflow's components, such as MLflow Projects and MLflow Modelsarea/sqlalchemy: Use of SQLAlchemy in the Tracking Service or Model Registryarea/windows: Windows supportLanguage
language/r: R APIs and clientslanguage/java: Java APIs and clientslanguage/new: Proposals for new client languagesIntegrations
integrations/azure: Azure and Azure ML integrationsintegrations/sagemaker: SageMaker integrationsintegrations/databricks: Databricks integrationsHow should the PR be classified in the release notes? Choose one:
rn/breaking-change- The PR will be mentioned in the "Breaking Changes" sectionrn/none- No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" sectionrn/feature- A new user-facing feature worth mentioning in the release notesrn/bug-fix- A user-facing bug fix worth mentioning in the release notesrn/documentation- A user-facing documentation change worth mentioning in the release notes