-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cron Tasks #42
Cron Tasks #42
Conversation
Co-authored-by: Megaklis Vasilakis <megaklis.vasilakis@gmail.com> Co-authored-by: Stavros Theocharis <stavrostheocharis@users.noreply.github.com>
"class2" | ||
] | ||
assert multi_metrics["accuracy"] == 0.6 | ||
assert multi_metrics.accuracy == 0.6 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How can we verify this number?
This question is more generic:
We should distinguish the tests that assert the results from the libraries used in our pipelines. Testing a pipeline does not necessarily mean testing the value that occurs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a nice discussion in general. Testing in ML may sometimes be hard. For such cases as the above you mention we can by ourselves do the calculations and check if the result is the same as the one indicated by the pipeline.
src/cron_tasks/monitoring_metrics.py
Outdated
data_drif_report = run_data_drift_pipeline(training_df, inference_df) | ||
# TODO: Fix pipeline to return a DataDriftTable first | ||
|
||
new_drifting_metric = entities.DriftingMetric( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be DriftingMetric from schemas?
Co-authored-by: John Zaras <sinnec@users.noreply.github.com> Co-authored-by: Charis Giaralis <gcharis@users.noreply.github.com> Co-authored-by: Stavros Theocharis <stavrostheocharis@users.noreply.github.com>
Co-authored-by: Megaklis Vasilakis megaklis.vasilakis@gmail.com
Co-authored-by: Stavros Theocharis stavrostheocharis@users.noreply.github.com