Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Data Drift View bugs #168090

Closed
valeriy42 opened this issue Oct 5, 2023 · 3 comments
Closed

[ML] Data Drift View bugs #168090

valeriy42 opened this issue Oct 5, 2023 · 3 comments
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:File and Index Data Viz ML file and index data visualizer :ml v8.11.0

Comments

@valeriy42
Copy link
Contributor

Hi @qn895,

great job on implementing the Data Drift view. I gave it a spin and noticed a couple of issues I wanted to document here:

  1. KS-Test value is missing when value ranges are disjunct. I used iris dataset and filtered two different classes for reference and comparison. Now petal_length doesn't show any Similarity p-value and "Drift detected" is noted as "No", although the values are completely different.
Screenshot 2023-10-05 at 13 33 06
  1. KS-Test p-value for pdays on the bank-marketing dataset is very small, although reference and comparison distributions are identical. I guess we have a numerical issue here we need to investigate.
image
  1. I really like how intuitive the interval selection is. However, when I was playing with the rulers on the farequote dataset they went out of sync.
image

Those are the only bugs I was able to notice. Overall, I think it is a great feature and it will be very useful for operationalizing ML in the stack.

@valeriy42 valeriy42 added bug Fixes for quality problems that affect the customer experience :ml labels Oct 5, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@peteharverson peteharverson added the Feature:File and Index Data Viz ML file and index data visualizer label Oct 5, 2023
@peteharverson peteharverson changed the title [ML] Data View bugs [ML] Data Drift View bugs Oct 5, 2023
@qn895
Copy link
Member

qn895 commented Oct 9, 2023

As mentioned in Slack, issue 1 and 2 require more investigation into the ES queries we are currently using. For the third issue with the ruler, we have an enhancement slated for 8.12 to revamp the user experience completely.

@qn895
Copy link
Member

qn895 commented Oct 19, 2023

Closing this issue as they have been fixed after #167911 and #168757

@qn895 qn895 closed this as completed Oct 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:File and Index Data Viz ML file and index data visualizer :ml v8.11.0
Projects
None yet
Development

No branches or pull requests

4 participants