Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add run links popover to metrics plot #2295

Merged
merged 40 commits into from Feb 28, 2020
Merged

Conversation

harupy
Copy link
Member

@harupy harupy commented Jan 12, 2020

What changes are proposed in this pull request?

Resolves #2279

How is this patch tested?

(Details)

Release Notes

Is this a user-facing change?

  • No. You can skip the rest of this section.
  • Yes. Give a description of this change to be included in the release notes for MLflow users.

(Details in 1-2 sentences. You can just refer to another PR with a description if this PR is part of a larger change.)

What component(s) does this PR affect?

  • UI
  • CLI
  • API
  • REST-API
  • Examples
  • Docs
  • Tracking
  • Projects
  • Artifacts
  • Models
  • Scoring
  • Serving
  • R
  • Java
  • Python

How should the PR be classified in the release notes? Choose one:

  • rn/breaking-change - The PR will be mentioned in the "Breaking Changes" section
  • rn/none - No description will be included. The PR will be mentioned only by the PR number in the "Small Bugfixes and Documentation Updates" section
  • rn/feature - A new user-facing feature worth mentioning in the release notes
  • rn/bug-fix - A user-facing bug fix worth mentioning in the release notes
  • rn/documentation - A user-facing documentation change worth mentioning in the release notes

@harupy
Copy link
Member Author

harupy commented Jan 12, 2020

The plot area gets initialized when the popover shows up. I'm wondering if it's possible to prevent the plot area from getting initialized by re-rendering.

demo

@harupy harupy changed the title Add runs link popover to metrics plot Add run links popover to metrics plot Jan 12, 2020
@juntai-zheng
Copy link
Collaborator

Hey @harupy, @smurching and I took a look at what was causing the issue. We've isolated it to the metrics prop changing in the MetricsPlotView, causing it to refresh whenever there is a click and the popover is created. This also happens when you're not zoomed in, though it's not noticeable because it doesn't change the graph. You can see how we found the bug in this screenshot:
image

We used the hook from here to log which prop was changed.

Do you think you could take a closer look as to why metrics is changing? If that doesn't change then the zoom in problem might be solved. Thanks!

@harupy
Copy link
Member Author

harupy commented Jan 23, 2020

@juntai-zheng Thanks for the investigation. The actual cause is when toggling the popover, the state popoverVisible changes. This change triggers re-rendering and the plot gets zoomed out.

@juntai-zheng
Copy link
Collaborator

@harupy ah sorry, forgot to mention that we had actually removed popoverVisible as another part of our investigation since we saw that it was causing re-rendering and didn't seem needed in MetricsPlotView. When we did, metrics was changing as well (causing the re-rendering).

@harupy
Copy link
Member Author

harupy commented Jan 24, 2020

@juntai-zheng Can you share the branch you're working on?

@juntai-zheng
Copy link
Collaborator

@harupy Here it is: https://github.com/juntai-zheng/mlflow/tree/popover

Just checking, the links created by the popover link me back to the MLflow home page. Is linking to the appropriate run a WIP?

@harupy
Copy link
Member Author

harupy commented Jan 27, 2020

@juntai-zheng

Is linking to the appropriate run a WIP?

Yes

@harupy
Copy link
Member Author

harupy commented Jan 27, 2020

@juntai-zheng

I checked your branch. In JavaScript, === compares identity for objects, not equality.

> const a = {};
> const b = {};
> const c = a;

> a === b
false

>>> a === c
true

The comparison below doesn't work when val is an object or a list.

prevProps[key] !== val

So the metrics wasn't changing.

@harupy
Copy link
Member Author

harupy commented Jan 28, 2020

One way to freeze the plot when toggling the popover is to use shouldComponentUpdate in MetricsPlotView (which is a stateless component) and prevent the re-render if the next props equal to the current ones.

@juntai-zheng
Copy link
Collaborator

Hi @harupy, you were correct in the hook being incorrect and metrics changing. We did some more digging and found out that because the state of the popover was contained in the state of MetricsPlotPanel, the changes to the state of the popover would also change the state of MetricsPlotPanel. Since MetricsPlotPanel is a parent to the popover and MetricsPlotView, whenever MetricsPlotPanel would re-render, so would MetricsPlotView. To avoid this, we made a ref to the popover in MetricsPlotPanel and only changed the state of RunLinksPopover, which subsequently only re-renders the popover and left the plot alone.

We made rough changes to my branch https://github.com/juntai-zheng/mlflow/tree/popover, so you can check it out from there.

@harupy
Copy link
Member Author

harupy commented Jan 28, 2020

@juntai-zheng Awesome. I'll reflect the change on this branch.

@harupy
Copy link
Member Author

harupy commented Jan 28, 2020

Now it looks like:

demo1

@harupy harupy marked this pull request as ready for review January 28, 2020 15:59
@juntai-zheng
Copy link
Collaborator

@harupy the popover looks great! Looks like a couple test are failing because of extra props being passed with our modifications. Do you think you could apply a fix for the tests?

@harupy
Copy link
Member Author

harupy commented Jan 29, 2020

@juntai-zheng I will fix the tests!

@gioa
Any thoughts on the popover design?

@harupy
Copy link
Member Author

harupy commented Feb 23, 2020

Added the following updates.

  • Display the popover on the left to prevent it from overlapping the tooltips on the plot.
  • Display metric values after run names.

latest

@harupy
Copy link
Member Author

harupy commented Feb 23, 2020

Maybe we should truncate run uuid in the run name (for example displaying only the first 8 characters) because the plot width is reduced when comparing multiple runs.

Screen Shot 2020-02-23 at 17 48 44

@Zangr
Copy link
Contributor

Zangr commented Feb 27, 2020

@harupy , the changes look merge ready, would you like to resolve the conflicts?

@harupy
Copy link
Member Author

harupy commented Feb 28, 2020

@Zangr Fixed the conflict!

@Zangr Zangr merged commit bc710d4 into mlflow:master Feb 28, 2020
@harupy harupy deleted the add-run-link-popup branch February 28, 2020 22:35
@harupy
Copy link
Member Author

harupy commented Feb 28, 2020

Maybe we should truncate run uuid in the run name (for example displaying only the first 8 characters) because the plot width is reduced when comparing multiple runs.

Screen Shot 2020-02-23 at 17 48 44

@Zangr What do you think about this?

@Zangr
Copy link
Contributor

Zangr commented Feb 29, 2020

Maybe we should truncate run uuid in the run name (for example displaying only the first 8 characters) because the plot width is reduced when comparing multiple runs.

Agree, I think we can show a short version and show the full id in the tooltip. cc @gioa

@harupy
Copy link
Member Author

harupy commented Feb 29, 2020

Yes, that makes sense. I'll open a PR.

@harupy
Copy link
Member Author

harupy commented Feb 29, 2020

@Zangr @smurching
I found a bug. The popover shows up after relayout by double click. I'll open a PR to fix this bug.

popover-bug

mparkhe added a commit that referenced this pull request Mar 1, 2020
#2493)

* modified README to include Model Registry component; augmented the Model Registry Workflow API with code snippets

* [mlflow/R] Update R client to call record logged model  (#2430)

* Update R client to call record logged model metadata with tracking server.

* fix

* Ran roxygen to update R docs.

* Updated R NAMESPACE to address cran check complaint.

* test fix

* fix test

* fix test

* Addressed review comment + removed forgotten debug string from earlier.

* [UI] Better error messages for model registry (#2456)

* typo: algorhitm->algorithm (#2490)

* Update model registry frontend JS with API changes (#2476)

* Save metric plot state in URLs for easier link sharing (#2393)

Metric plot config is now saved in the URL to simplify sharing plots & collaborating

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

+1

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Good idea to have an actor do the task!

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

thanks for catching the subject-verb agreement!

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update README.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Fix bar chart bug introduced in #2393 (#2498)

* minor change and incorporated suggestions

* minor typo

* mlflow gc command (#2265)

* Added hard delete run in sql alchemy store. Added delete artifacts in hdfs store. Added hard delete run endpoint and function in client.

* Fixed delete in hdfs artifact repo. Fixed hard delete endpoint.

* Add support for file store

* Artifacts are deleted on client side when using hard-delete run cli

* Added support for local_artifact_repo

* Fix linter

* Added test for fluent api

* Fix tests

* Added not implemented yet exception for artifact stores other than hdfs and local

* Removed hard delete run endpoint and exposition in client.py

* Added gc command line

* Fix tests

* Remove hard delete runs command line

* Add method _get_deleted_runs in sql alchemy store

* Added test for _get_deleted_runs in sql alchemy store

* Fix linter

* Fix linter

* Added comments to _hard_delete_run and to the gc cli.

* Add implementation of get_deleted_runs for file_store

* delete_artifacts now take a relative path. get_deleted_runs in sql alchemry store returns a list of string. Changes to comments.

* Fix linter

* Fix tests

* Added end to end test for mlflow gc command line

* Fix windows tests

* Fix hdfs_artifact_repo tests on windows

* Fix test_cli for python 2

* Fix linter

* Fix python 2 tests

* Fix python 2 tests

* Retrigeer tests becaused of failed R tests

* Changed urllib import to use six.moves

* Add run links popover to metrics plot (#2295)

* Add runs link popover

* Rename

* Trigger travis

* Use ref and refactor

* Add onRelayout

* Remove unused functions and state

* Refactor

* Add comment

* Fix test

* Fix prop type

* Rename var

* Fix prop type

* Add test for RunLinksPopover

* Add icon

* Refactor

* Close popover when Escaoe key pressed

* use single quotes

* Use named export

* Do not expose state

* Resolve conflict

* Fix popover to use props

* Remove unnecessary comment

* Remove unnecessary comment

* Fix test

* Rename popover title

* Fix style of close button

* Add sorting by y

* Display the popover on the left

* Display metric value

* fix

* Fix

* Fix

* Remove runUuid

* Fix test

* Fix tests

* nit

* API cleanup (#2502)

* API cleanup

removing `stage` field from `UpdateModelVersion` proto

* Minor docs tweaks for MLflow gc command (#2506)

* Added hard delete run in sql alchemy store. Added delete artifacts in hdfs store. Added hard delete run endpoint and function in client.

* Fixed delete in hdfs artifact repo. Fixed hard delete endpoint.

* Add support for file store

* Artifacts are deleted on client side when using hard-delete run cli

* Added support for local_artifact_repo

* Fix linter

* Added test for fluent api

* Fix tests

* Added not implemented yet exception for artifact stores other than hdfs and local

* Removed hard delete run endpoint and exposition in client.py

* Added gc command line

* Fix tests

* Remove hard delete runs command line

* Add method _get_deleted_runs in sql alchemy store

* Added test for _get_deleted_runs in sql alchemy store

* Fix linter

* Fix linter

* Added comments to _hard_delete_run and to the gc cli.

* Add implementation of get_deleted_runs for file_store

* delete_artifacts now take a relative path. get_deleted_runs in sql alchemry store returns a list of string. Changes to comments.

* Fix linter

* Fix tests

* Added end to end test for mlflow gc command line

* Fix windows tests

* Fix hdfs_artifact_repo tests on windows

* Fix test_cli for python 2

* Fix linter

* Fix python 2 tests

* Fix python 2 tests

* Retrigeer tests becaused of failed R tests

* Add/improve documentation

* Fix indent

* resolved and incorporate suggestions

* Fixing JS reducer on model version delete (#2507)

* resolved and addressed suggestions

* fixing search test (unrelated to documentation)

* fix docs

Co-authored-by: tomasatdatabricks <33237569+tomasatdatabricks@users.noreply.github.com>
Co-authored-by: Aaron Davidson <aaron@databricks.com>
Co-authored-by: Rens <rens@dimmendaal.com>
Co-authored-by: Siddharth Murching <smurching@gmail.com>
Co-authored-by: Stephanie Bodoff <stephanie.bodoff@databricks.com>
Co-authored-by: t-henri <51711739+t-henri@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: Mani Parkhe <mani@databricks.com>
Co-authored-by: dbczumar <39497902+dbczumar@users.noreply.github.com>
@smurching smurching added the rn/feature Mention under Features in Changelogs. label Mar 2, 2020
avflor pushed a commit to avflor/mlflow that referenced this pull request Aug 22, 2020
* Add runs link popover

* Rename

* Trigger travis

* Use ref and refactor

* Add onRelayout

* Remove unused functions and state

* Refactor

* Add comment

* Fix test

* Fix prop type

* Rename var

* Fix prop type

* Add test for RunLinksPopover

* Add icon

* Refactor

* Close popover when Escaoe key pressed

* use single quotes

* Use named export

* Do not expose state

* Resolve conflict

* Fix popover to use props

* Remove unnecessary comment

* Remove unnecessary comment

* Fix test

* Rename popover title

* Fix style of close button

* Add sorting by y

* Display the popover on the left

* Display metric value

* fix

* Fix

* Fix

* Remove runUuid

* Fix test

* Fix tests

* nit
avflor pushed a commit to avflor/mlflow that referenced this pull request Aug 22, 2020
mlflow#2493)

* modified README to include Model Registry component; augmented the Model Registry Workflow API with code snippets

* [mlflow/R] Update R client to call record logged model  (mlflow#2430)

* Update R client to call record logged model metadata with tracking server.

* fix

* Ran roxygen to update R docs.

* Updated R NAMESPACE to address cran check complaint.

* test fix

* fix test

* fix test

* Addressed review comment + removed forgotten debug string from earlier.

* [UI] Better error messages for model registry (mlflow#2456)

* typo: algorhitm->algorithm (mlflow#2490)

* Update model registry frontend JS with API changes (mlflow#2476)

* Save metric plot state in URLs for easier link sharing (mlflow#2393)

Metric plot config is now saved in the URL to simplify sharing plots & collaborating

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

+1

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Good idea to have an actor do the task!

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

thanks for catching the subject-verb agreement!

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update docs/source/model-registry.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Update README.rst

Co-Authored-By: Stephanie Bodoff <stephanie.bodoff@databricks.com>

* Fix bar chart bug introduced in mlflow#2393 (mlflow#2498)

* minor change and incorporated suggestions

* minor typo

* mlflow gc command (mlflow#2265)

* Added hard delete run in sql alchemy store. Added delete artifacts in hdfs store. Added hard delete run endpoint and function in client.

* Fixed delete in hdfs artifact repo. Fixed hard delete endpoint.

* Add support for file store

* Artifacts are deleted on client side when using hard-delete run cli

* Added support for local_artifact_repo

* Fix linter

* Added test for fluent api

* Fix tests

* Added not implemented yet exception for artifact stores other than hdfs and local

* Removed hard delete run endpoint and exposition in client.py

* Added gc command line

* Fix tests

* Remove hard delete runs command line

* Add method _get_deleted_runs in sql alchemy store

* Added test for _get_deleted_runs in sql alchemy store

* Fix linter

* Fix linter

* Added comments to _hard_delete_run and to the gc cli.

* Add implementation of get_deleted_runs for file_store

* delete_artifacts now take a relative path. get_deleted_runs in sql alchemry store returns a list of string. Changes to comments.

* Fix linter

* Fix tests

* Added end to end test for mlflow gc command line

* Fix windows tests

* Fix hdfs_artifact_repo tests on windows

* Fix test_cli for python 2

* Fix linter

* Fix python 2 tests

* Fix python 2 tests

* Retrigeer tests becaused of failed R tests

* Changed urllib import to use six.moves

* Add run links popover to metrics plot (mlflow#2295)

* Add runs link popover

* Rename

* Trigger travis

* Use ref and refactor

* Add onRelayout

* Remove unused functions and state

* Refactor

* Add comment

* Fix test

* Fix prop type

* Rename var

* Fix prop type

* Add test for RunLinksPopover

* Add icon

* Refactor

* Close popover when Escaoe key pressed

* use single quotes

* Use named export

* Do not expose state

* Resolve conflict

* Fix popover to use props

* Remove unnecessary comment

* Remove unnecessary comment

* Fix test

* Rename popover title

* Fix style of close button

* Add sorting by y

* Display the popover on the left

* Display metric value

* fix

* Fix

* Fix

* Remove runUuid

* Fix test

* Fix tests

* nit

* API cleanup (mlflow#2502)

* API cleanup

removing `stage` field from `UpdateModelVersion` proto

* Minor docs tweaks for MLflow gc command (mlflow#2506)

* Added hard delete run in sql alchemy store. Added delete artifacts in hdfs store. Added hard delete run endpoint and function in client.

* Fixed delete in hdfs artifact repo. Fixed hard delete endpoint.

* Add support for file store

* Artifacts are deleted on client side when using hard-delete run cli

* Added support for local_artifact_repo

* Fix linter

* Added test for fluent api

* Fix tests

* Added not implemented yet exception for artifact stores other than hdfs and local

* Removed hard delete run endpoint and exposition in client.py

* Added gc command line

* Fix tests

* Remove hard delete runs command line

* Add method _get_deleted_runs in sql alchemy store

* Added test for _get_deleted_runs in sql alchemy store

* Fix linter

* Fix linter

* Added comments to _hard_delete_run and to the gc cli.

* Add implementation of get_deleted_runs for file_store

* delete_artifacts now take a relative path. get_deleted_runs in sql alchemry store returns a list of string. Changes to comments.

* Fix linter

* Fix tests

* Added end to end test for mlflow gc command line

* Fix windows tests

* Fix hdfs_artifact_repo tests on windows

* Fix test_cli for python 2

* Fix linter

* Fix python 2 tests

* Fix python 2 tests

* Retrigeer tests becaused of failed R tests

* Add/improve documentation

* Fix indent

* resolved and incorporate suggestions

* Fixing JS reducer on model version delete (mlflow#2507)

* resolved and addressed suggestions

* fixing search test (unrelated to documentation)

* fix docs

Co-authored-by: tomasatdatabricks <33237569+tomasatdatabricks@users.noreply.github.com>
Co-authored-by: Aaron Davidson <aaron@databricks.com>
Co-authored-by: Rens <rens@dimmendaal.com>
Co-authored-by: Siddharth Murching <smurching@gmail.com>
Co-authored-by: Stephanie Bodoff <stephanie.bodoff@databricks.com>
Co-authored-by: t-henri <51711739+t-henri@users.noreply.github.com>
Co-authored-by: Harutaka Kawamura <hkawamura0130@gmail.com>
Co-authored-by: Mani Parkhe <mani@databricks.com>
Co-authored-by: dbczumar <39497902+dbczumar@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rn/feature Mention under Features in Changelogs.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FR][UI] Enable clicking through to individual runs from metric plots
5 participants