Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Task]: Only handle nulls for nullable columns in pandas to Beam row conversion #22948

Open
TheNeuralBit opened this issue Aug 29, 2022 · 0 comments
Open

[Task]: Only handle nulls for nullable columns in pandas to Beam row conversion #22948

TheNeuralBit opened this issue Aug 29, 2022 · 0 comments
Labels
dataframe dsl P2 task

Comments

@TheNeuralBit
Copy link
Member

What needs to happen?

Currently when we convert a pandas DataFrame to Beam Rows we check for nulls in every column. This is wasteful, we should instead only do null-checking for the columns that we know to be nullable.

Issue Priority

Priority: 2

Issue Component

Component: dsl-dataframe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
dataframe dsl P2 task
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@TheNeuralBit @damccorm