Skip to content

add check_y and check_x_y to check that indexes of X and y match when encoding - #410

Merged
solegalli merged 10 commits into
mainfrom
checkky
Apr 14, 2022
Merged

add check_y and check_x_y to check that indexes of X and y match when encoding#410
solegalli merged 10 commits into
mainfrom
checkky

Conversation

@solegalli

@solegalli solegalli commented Apr 7, 2022

Copy link
Copy Markdown
Collaborator

closes #377
closes #376
closes #365

@noahjgreen295 @bmreiniger

I am bringing the changes from #377 to this new PR at the back of former PR #408

Regarding the X and y checks, I am bringing in sklearn functionality to check y and then to check X and y (taken from their check_X_y function). I think that decreases the amount of new things we need to code.

I think I captured most of what we discussed in #377 , and I brought forward the tests and ideas from @noahjgreen295 as they were or slightly reformatted and also the suggestions from @bmreiniger.

Would you guys have a look and let me know what you think?

Let me know if you are OK to merge.

Thank you so much for raising and also for the support on this issue.

@noahjgreen295

Copy link
Copy Markdown
Contributor

Hi @solegalli - I'm sorry I'm just having git some trouble getting this branch into my cloned repo. I did the following:
% git pull upstream checkky
% git switch checkky
I'm pretty sure that what I have now locally is the correct branch (if not, let me know).
I am now trying to put a copy into my cloned repo, to push changes to there, but when I try git push, I get this error:
remote: Permission to feature-engine/feature_engine.git denied to noahjgreen295. fatal: unable to access 'https://github.com/feature-engine/feature_engine.git/': The requested URL returned error: 403

Am I doing this the right way? Sorry for the basic git question - thanks!

@bmreiniger bmreiniger left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going to suggest using sklearn's check_X_y and/or check_array in a fashion similar to your usage of _check_y (could reduce some duplication in your check_X, and uses a public sklearn method instead of a private one), but there's a lot going on there for converting to array/sparse that would be a waste. Hrm...

Otherwise, minor comments.

Comment on lines +161 to +165
def _check_X_y(X, y):
X = check_X(X)
y = check_y(y, multi_output=multi_output, y_numeric=y_numeric)
check_consistent_length(X, y)
return X, y

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In every case below you call this function, so consider just running this code first without defining it as a local function.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, makes sense. The thing is, if I ran this function first, then X and y would be a pandas df and series, so I can't really check for the different scenarios any more. That's why I took this convoluted approach instead.

How would you approach it?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, of course. This is fine then. You could also maybe define "the" index in cases, then do the checks/conversions?

Comment thread feature_engine/dataframe_checks.py Outdated
Parameters
----------
X: Pandas DataFrame or numpy ndarray
y: Pandas Series or numpy ndarray

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some parameters not documented

validated as a vector. y cannot have np.nan or np.inf values if
multi_output=True.

y_numeric : bool, default=False

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

default in docstring disagrees with code default

Comment thread feature_engine/dataframe_checks.py Outdated
raise ValueError("y cannot be None.")

elif isinstance(y, pd.Series):
_check_y(y, multi_output=multi_output, y_numeric=y_numeric)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by not assigning, you get all the checks without the conversion, but e.g. the numeric conversion also doesn't happen...

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thank you! Refactoring.

@solegalli

Copy link
Copy Markdown
Collaborator Author

Hi @solegalli - I'm sorry I'm just having git some trouble getting this branch into my cloned repo. I did the following: % git pull upstream checkky % git switch checkky I'm pretty sure that what I have now locally is the correct branch (if not, let me know). I am now trying to put a copy into my cloned repo, to push changes to there, but when I try git push, I get this error: remote: Permission to feature-engine/feature_engine.git denied to noahjgreen295. fatal: unable to access 'https://github.com/feature-engine/feature_engine.git/': The requested URL returned error: 403

Am I doing this the right way? Sorry for the basic git question - thanks!

You need to push the branch to your own repo, and make a PR from there to the main repo's checkky branch.

@solegalli solegalli changed the title adds check_y, and check_x_y, checks that indexes of X and y match when encoding add check_y and check_x_y to check that indexes of X and y match when encoding Apr 14, 2022
@solegalli
solegalli merged commit 5d45d91 into main Apr 14, 2022
@solegalli
solegalli deleted the checkky branch April 14, 2022 05:07
@solegalli

Copy link
Copy Markdown
Collaborator Author

Thank you @bmreiniger @noahjgreen295 for raising and your support on this issue. Merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants