Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEAT][tabular][CV] Label-prediction distribution comparison #1444

Open
nirhutnik opened this issue May 15, 2022 · 0 comments
Open

[FEAT][tabular][CV] Label-prediction distribution comparison #1444

nirhutnik opened this issue May 15, 2022 · 0 comments
Labels
ds Tasks suited for Data Scientists feature Feature update or code change to the package good first issue Good for newcomers
Milestone
Short-term Backlog

Comments

@nirhutnik
Copy link
Contributor

nirhutnik commented May 15, 2022 •
edited

Create a check, both in tabular and in vision that compares the disribution of the prediction and the label. This is a sanity check that the model predicts the label well enough.
Use drift methods (PSI / Cramer's V and EMD) to compare.
Find a good name for this check.

In vision - use properties
@nirhutnik nirhutnik added the feature Feature update or code change to the package label May 15, 2022
@nirhutnik nirhutnik added this to the 0.6.6 milestone May 15, 2022
@ItayGabbay ItayGabbay modified the milestones: 0.7.1, 0.8 May 23, 2022
@noamzbr noamzbr added this to the 0.9 milestone Jul 13, 2022
@noamzbr noamzbr added the ds Tasks suited for Data Scientists label Jul 13, 2022
@ItayGabbay ItayGabbay added linear and removed linear labels Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
ds Tasks suited for Data Scientists feature Feature update or code change to the package good first issue Good for newcomers
Projects
None yet
Milestone
Short-term Backlog
Development

No branches or pull requests
3 participants
@noamzbr @ItayGabbay @nirhutnik