-
Notifications
You must be signed in to change notification settings - Fork 163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Numerical columns treated as categorical #12
Comments
Hi Alex, It would be great if you can share your data. You can choose any method that works for you eg a Google Drive Link, other file upload, send us the original source, ... There is definitely some work to be done from our side to further clarify or catch the errors that are currently raised from sklearn. The error "'DataFrame' object has no attribute 'dtype'" seems strange to me and it seems like internally .dtype is trying to be called on a DataFrame instead of a series. Is it possible that there are columns with the same name? By the way, why are you trying to predict YearBuilt with all columns (x is YearBuilt) ? Don't you want to do the opposite? And try to predict YearBuilt with the other columns? (y is YearBuilt)? Cheers, |
Hi Florian ! Thx for your answer. I've uploaded the csv and jupyter notebook to Google Drive. Here is the link : https://drive.google.com/open?id=127BwkUTcKF18_Kh599jHlsMVIFnpwU3S The original data is available on Kaggle here : The data I am using was obtained from the original by combining the 2015 and 2016 tables and cleaning them. Yes, I also found it strange that the .dtype looks like it's called on a DataFrame. It's particularly weird as that happens only wen task is specified, not when task = None. The columns all have a unique name. Using 'YearBuilt' to predict was just a test ! I wanted to understand which variable was being considered a categorical variable. My initial objective was to calculate the whole pps matrix and compare it to the correlation matrix to see if it could provide more insight, like you did in your article. Cheers, Alexander |
Hi Alexander, thank you for sending over the data. The first error appeared because the logic with overriding the categorical breakpoint does not seem to work. I will have to look into this again. It failed for the following code: The second error is due to the fact that the for-loop resulted in the following: Cheers, |
Hi Florian, I'm happy to learn the data helped you identify the problems :) I had a hint the categorical breakpoint might not work but couldn't be sure as the for loop was acting weird. Didn't anticipate the x = y exception ! Thanks again for providing this package and taking the time to update and support it. Cheers, Alexander |
Yes, the data was very helpful - thank you for that! |
There have been two issues here:
|
Hi guys,
I heard of PPS, through your article and was curious to test it. I have tried implementing it on some data I've been working on.
Unfortunately, I get numerous error messages when calculating the pps matrix :
My guess is pps is considering my data to be categorical and therefore trying to apply classification with a huge number of labels.
Looking at how pps determines if the data is numerical or categorical, I cannot find the reason it would consider my data categorical :
Also, if I try to force the pps score to be calculated using task = 'regression', I get the following error :
Here is my code :
Is there something I am missing ? If not, would you like me to share the data with you ? (I do not know which sharing method is more convenient for you)
The text was updated successfully, but these errors were encountered: