Skip to content

Cleanup of "Log metrics over time" notebook in preparation for docs#301

Merged
validbeck merged 8 commits intomainfrom
beck/sc-8186/documentation-support-threshold-lines-in
Jan 30, 2025
Merged

Cleanup of "Log metrics over time" notebook in preparation for docs#301
validbeck merged 8 commits intomainfrom
beck/sc-8186/documentation-support-threshold-lines-in

Conversation

@validbeck
Copy link
Collaborator

@validbeck validbeck commented Jan 28, 2025

Internal Notes for Reviewers

sc-8186

1st phase for the documentation of being able to set unit metric thresholds. I cleaned up our "Log metric over time" notebook so I can reference it in our documentation site: https://github.com/validmind/validmind-library/blob/beck/sc-8186/documentation-support-threshold-lines-in/notebooks/how_to/log_metrics_over_time.ipynb

Part 2 is slotting in a reference to it here on the Work with metrics over time guide replacing the "Intro to Unit Metrics" link: validmind/documentation#633

@validbeck validbeck added the internal Not to be externalized in the release notes label Jan 28, 2025
@validbeck validbeck self-assigned this Jan 28, 2025
@validbeck validbeck requested a review from juanmleng January 28, 2025 21:30
Copy link
Contributor

@juanmleng juanmleng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks great Beck! I just left a tiny suggestion.

@validbeck validbeck force-pushed the beck/sc-8186/documentation-support-threshold-lines-in branch from 8d38ffa to caf2686 Compare January 30, 2025 17:23
@github-actions
Copy link
Contributor

PR Summary

This pull request enhances the log_metrics_over_time.ipynb notebook by providing a more detailed guide on how to use the ValidMind Library for logging and visualizing model performance metrics over time. The changes include:

  • Improved markdown documentation to better explain the process of tracking and visualizing key model performance metrics such as AUC, F1 score, precision, recall, and accuracy using ValidMind.
  • Added a table of contents for easier navigation within the notebook.
  • Introduced detailed explanations and examples for using the log_metric() function to log metrics over time, including setting custom thresholds and visualizing these metrics in the ValidMind Platform.
  • Provided code snippets to demonstrate how to initialize the ValidMind Library, load model credentials, and log multiple metrics with custom thresholds.
  • Enhanced the narrative to guide users through the process of setting up ongoing monitoring and using the ValidMind Platform's visualization tools.

These changes aim to improve the user experience by offering a comprehensive guide to monitoring model performance and detecting potential issues through systematic metric logging.

Test Suggestions

  • Run the notebook to ensure all code cells execute without errors.
  • Verify that the table of contents links navigate to the correct sections within the notebook.
  • Test the log_metric() function with various metrics and thresholds to ensure they are logged correctly.
  • Check the visualization of metrics in the ValidMind Platform to confirm that the metrics are displayed as expected.
  • Simulate a scenario with multiple metrics and thresholds to observe the behavior of the visualization over time.

@validbeck validbeck merged commit 6d6b4e7 into main Jan 30, 2025
6 checks passed
@validbeck validbeck deleted the beck/sc-8186/documentation-support-threshold-lines-in branch January 30, 2025 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants