Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add special case to check values of str column #808

Open
3 tasks done
cosmicBboy opened this issue Mar 29, 2022 · 0 comments
Open
3 tasks done

Add special case to check values of str column #808

cosmicBboy opened this issue Mar 29, 2022 · 0 comments
Labels
bug Something isn't working

Comments

@cosmicBboy
Copy link
Collaborator

Describe the bug

FYI @jeffzi @dineshkumar-23

The bug is clearly described here. Basically, since str dtype arrays are translated to a numpy object arrays, any object can exist within such a column and still pass validation.

There is now pandas.StringDtype since pandas > 1.0, but I think it's still important to special-case this type because (i) many users may not be aware of it and (ii) I think pandera should start getting into the business of correcting some of pandas' quirks, esp. when it comes to the type system.

The special-casing should be implemented at the DataType definition (i.e. pandera.engines.numpy_engine.String) after we have an API for logical data types #798.

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of pandera.
  • (optional) I have confirmed this bug exists on the master branch of pandera.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

See #807

Expected behavior

Failure cases of non-string objects in a numpy object array (aka a string column) should be correctly reported.

@cosmicBboy cosmicBboy added the bug Something isn't working label Mar 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant