Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[API] Add model endpoint list metrics/results endpoint #5484

Merged
merged 20 commits into from
May 5, 2024
Merged

[API] Add model endpoint list metrics/results endpoint #5484

merged 20 commits into from
May 5, 2024

Conversation

jond01
Copy link
Member

@jond01 jond01 commented May 1, 2024 •
edited by jira bot

Resolves ML-6287.

Currently, support only:

  • results - metrics are not reported and saved yet,
  • V3IO - to be extended to MySQL later.

The core functionality is unit-tested here and was checked e2e by me. For example:
image

@jond01 jond01 requested a review from gtopper May 2, 2024 07:57
gtopper
gtopper reviewed May 2, 2024
View reviewed changes
mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/writer.py Show resolved Hide resolved
server/api/api/endpoints/model_monitoring.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Outdated Show resolved Hide resolved
@jond01 jond01 requested a review from gtopper May 2, 2024 14:11
jond01 and others added 17 commits May 4, 2024 22:13
@jond01
Move a log to its place
3780e81
@jond01
Rename alert "id" to "model_endpoint"
6fd5b5d
@jond01
Move model_monitoring_access_key to commons
debb654 
Eliminate the code duplication
@jond01
Make access_key optional in KVStoreBase
0984f82
@jond01
Add str method to ModelMonitoringAppLabel
c111315
@jond01
Add ModelEndpointMonitoringMetric pydantic model
78fa505
@jond01
Add API endpoint skeleton
9cebb92
@jond01
Implement the metrics extraction
c47b90e
@jond01
Test _extract_metrics_from_items
5acde91
@jond01
Test get_model_endpoint_metrics
fa6f33c
@jond01
Move the query param "type" to the right place
da15c64 
See https://fastapi.tiangolo.com/tutorial/query-params/
@jond01 @gtopper
Update mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py
47f0609 
Co-authored-by: Gal Topper <gal.topper@gmail.com>
@jond01
Allow only 404 not found http errors from v3io
aea94ba 
Add a test
@jond01
Rename items -> app_items
642e13c
@jond01
Fix comment
b555d56
@jond01
Add project to the metric obj and use period as joiner
9a531ac
@jond01
Add a TODO comment - use V3IO aio python client
b112054
@jond01 jond01 force-pushed the task/add-app-results-endpoint branch from 4c70829 to b112054 Compare May 4, 2024 19:54
gtopper
gtopper reviewed May 5, 2024
View reviewed changes
Copy link
Collaborator

@gtopper gtopper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, only a couple of comments.

mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Outdated Show resolved Hide resolved
mlrun/model_monitoring/db/stores/v3io_kv/kv_store.py Show resolved Hide resolved
@jond01 jond01 requested a review from gtopper May 5, 2024 07:34
@assaf758 assaf758 merged commit ecff614 into mlrun:development May 5, 2024
10 checks passed
@jond01 jond01 deleted the task/add-app-results-endpoint branch May 5, 2024 09:00
@jond01 jond01 mentioned this pull request May 8, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gtopper gtopper gtopper approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@jond01 @gtopper @assaf758