[Model Monitoring] Add MySQL support for saving and querying metrics #5641
Conversation
Remove the previous dummy test
Use a shared function and substitute the cases
The last analyzed should not be empty
It doesn't work otherwise (I did many tries): ``` NotImplementedError: This backend does not support multiple-table criteria within DELETE ```
This removal unnecessitates the type casting
…-metrics-and-query # Conflicts: # mlrun/model_monitoring/db/stores/sqldb/sql_store.py
…-metrics-and-query # Conflicts: # mlrun/model_monitoring/db/stores/sqldb/sql_store.py
…-query # Conflicts: # mlrun/model_monitoring/db/stores/sqldb/models/__init__.py # mlrun/model_monitoring/db/stores/sqldb/sql_store.py # tests/model_monitoring/test_stores/test_sql.py
jond01
changed the title
/metrics endpoint
| ) | ||
|
|
||
| return [ | ||
| mm_schemas.ModelEndpointMonitoringMetric( |
There was a problem hiding this comment.
I’m not sure if this is the right PR, but yesterday you mentioned that the result kind is being returned to the UI. I don't see it here. I noticed it’s being returned from the TSDB, but it seems more appropriate to return it from the /metrics endpoint since it’s metadata.
There was a problem hiding this comment.
"kind" isn't designed to be in this response, see section 2-c in the description of ML-6119.
There was a problem hiding this comment.
The design can be changed. Don't you think it's more accurate the way I mentioned?
There was a problem hiding this comment.
I think that's a valid point. I would also like to send the latest status in case of a result, to indicate which results should be inspected.
However, we discussed showing the status in the dropdown, and we deferred it (to >=1.8), so I think we shouldn't reopen now the API.
I believe it might not be that big here 😉 |
Implements ML-6268.
I tested the
/metricsendpoint on a system configured to use MySQL in model monitoring, and it works:Example
[ { "project": "test-app-flow-sql-3", "app": "evidently-app-test-v2", "type": "result", "name": "data_drift_test", "full_name": "test-app-flow-sql-3.evidently-app-test-v2.result.data_drift_test" }, { "project": "test-app-flow-sql-3", "app": "histogram-data-drift", "type": "result", "name": "general_drift", "full_name": "test-app-flow-sql-3.histogram-data-drift.result.general_drift" }, { "project": "test-app-flow-sql-3", "app": "monitoring-test-v2", "type": "result", "name": "data_drift_test", "full_name": "test-app-flow-sql-3.monitoring-test-v2.result.data_drift_test" }, { "project": "test-app-flow-sql-3", "app": "monitoring-test-v2", "type": "result", "name": "model_perf", "full_name": "test-app-flow-sql-3.monitoring-test-v2.result.model_perf" }, { "project": "test-app-flow-sql-3", "app": "histogram-data-drift", "type": "metric", "name": "hellinger_mean", "full_name": "test-app-flow-sql-3.histogram-data-drift.metric.hellinger_mean" }, { "project": "test-app-flow-sql-3", "app": "histogram-data-drift", "type": "metric", "name": "kld_mean", "full_name": "test-app-flow-sql-3.histogram-data-drift.metric.kld_mean" }, { "project": "test-app-flow-sql-3", "app": "histogram-data-drift", "type": "metric", "name": "tvd_mean", "full_name": "test-app-flow-sql-3.histogram-data-drift.metric.tvd_mean" }, { "project": "test-app-flow-sql-3", "app": "mlrun-infra", "type": "metric", "name": "invocations", "full_name": "test-app-flow-sql-3.mlrun-infra.metric.invocations" } ]