Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type check fails for complex types #4

Open
monkey0head opened this issue Dec 26, 2022 · 0 comments
Open

Type check fails for complex types #4

monkey0head opened this issue Dec 26, 2022 · 0 comments

Comments

@monkey0head
Copy link
Collaborator

monkey0head commented Dec 26, 2022

If complex type is present in schema (e.g. ArrayType), it could not be compared using symmetric_difference as done here

def _check_names_and_types(df1_schema, df2_schema):
as the type is not hashable.
Need to rework this code part to compare only names and types (do not compare nullability) and be able to work with complex types.
Proposed: sort two lists with (name, DataType) as done now and compare columns one-by-one. Raise error on the first difference.

Add test showing the functionality works for simple and complex types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant