Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Model Monitoring] Improvements to mm-apps do_tracking API #5474

Merged
merged 34 commits into from
May 5, 2024

Conversation

davesh0812
Copy link
Contributor

@davesh0812 davesh0812 commented Apr 30, 2024

Implements - ML-6259.

  1. Implements ModelMonitoringApplicationBaseV2 :
    a. with new do_tracing signature.
    b. delete self.context and uses the monitoring_context param as mlrun context.
  2. Implements MonitoringApplicationContext - The monitoring context holds all the relevant information for the monitoring application, and also it can be used for logging artifacts and results.
  3. Implements EvidentlyModelMonitoringApplicationBaseV2.
  4. Add __PrepareMonitoringEvent step the application serving graph - preparing the application event for the application step.
  5. Update _PushToWriter step.
  6. Update HistogramDataDriftApplication to v2.
  7. Change ModelMonitoringWriter and ModelMonitoringWriter._reconstruct_event to work with 2 kinds of events - metric and result, for now the ModelMonitoringWriter writes only the results to the databases.
  8. Change the applications (regular and evidently) and the result object directories.

@davesh0812 davesh0812 marked this pull request as draft April 30, 2024 21:13
@davesh0812 davesh0812 marked this pull request as ready for review May 2, 2024 09:30
@jond01 jond01 self-requested a review May 2, 2024 14:53
Copy link
Member

@jond01 jond01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

I had a few comments about the docs, tests, and implementation.

mlrun/model_monitoring/applications/application_steps.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/applications/base.py Show resolved Hide resolved
mlrun/model_monitoring/applications/results.py Outdated Show resolved Hide resolved
tests/system/model_monitoring/test_app.py Outdated Show resolved Hide resolved
@davesh0812 davesh0812 requested a review from jond01 May 5, 2024 10:25
Copy link
Member

@Eyal-Danieli Eyal-Danieli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Few comments and suggestions, mostly docstrings and typehints :)

mlrun/model_monitoring/applications/_application_steps.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/applications/base.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/applications/base.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/applications/results.py Outdated Show resolved Hide resolved
@assaf758 assaf758 merged commit f942608 into mlrun:development May 5, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants