Skip to content

Support direct polars scan#755

Merged
changhiskhan merged 5 commits intomainfrom
changhiskhan/pickle-ds
Apr 6, 2023
Merged

Support direct polars scan#755
changhiskhan merged 5 commits intomainfrom
changhiskhan/pickle-ds

Conversation

@changhiskhan
Copy link
Copy Markdown
Contributor

Closes #754

return self.uri, self._ds.version()

def __setstate__(self, state):
self._uri, version = state
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the fix? does polas have issues with picking / distributed pickle to different machines? I am not certain now, whether _Dataset has any OS resource that can not be pickled / unpickled from another machine.

Not a blocker tho.

@changhiskhan changhiskhan merged commit 24e107b into main Apr 6, 2023
@changhiskhan changhiskhan deleted the changhiskhan/pickle-ds branch April 6, 2023 20:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

direct polars scan is broken

2 participants