Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Minimum Observation Count for IV value? #3

Open
Ishankhare1 opened this issue Apr 30, 2018 · 2 comments
Open

Minimum Observation Count for IV value? #3

Ishankhare1 opened this issue Apr 30, 2018 · 2 comments

Comments

@Ishankhare1
Copy link

Hi Thomas,

I am getting zero information value for many variables having low observation count. I was getting significant IV value using 'Information' package in R for these variables. What is the cutoff for minimum observation count to get non-zero IV value in this package? Is it minimum 5% observations in each bucket?

Regards,
Ishan

@tomasgreif
Copy link
Owner

What data do you have. In case it is heavily skewed it might be some issue. Or try to use different rpart control in case this is in fact using decision tree to make the splits.

@Ishankhare1
Copy link
Author

Information Package Results:

Sr | Independent_Var_2 | N | Percent | WOE | IV
1 | [0,0] | 23921 | 0.946504174 | -0.654004424 | 0.300204339
2 | [0,1] | 1352 | 0.053495826 | 2.441050188 | 1.420707247

Tomasgreif Package Results:

Variable | InformationValue | Strength | class | outcome_0 | outcome_1 | woe | miv
Independent_Var_2 | 0 | Wery weak | (;) | 24738 | 535 | 0 | 0

As you see the IV from information package is 1.42 and from this package it is 0.00. I cant share actual data but you can populate as its binary. The total non zero independent var_2 is 5.3% so I was thinking is there a cutoff in this package for minimum observations in a bin?

Below is the data:

Dependent Var_1 Independent Var_2 Frequency
0 0 1083
1 0 269
0 1 23655
1 1 266

Kindly let me know if i should use the results of this package or refer to information package?

Ishan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants