Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade pyarrow #245

Closed
randyzwitch opened this issue Jul 8, 2019 · 4 comments
Closed

Upgrade pyarrow #245

randyzwitch opened this issue Jul 8, 2019 · 4 comments

Comments

@randyzwitch
Copy link
Contributor

Evaluate whether we can loosen the upper bound of the pyarrow dependency

@randyzwitch
Copy link
Contributor Author

We do not currently support pyarrow 0.14, as some breaking changes were made between 0.13 and 0.14. Specifically, the following passage could be our issue, which is resolved directly in OmniSciDB:

The dictionary API has been reworked in ARROW-3144. The dictionary values used to be tied to the DictionaryType instance, which ended up too inflexible. Since dictionary-encoding is more often an optimization than a semantic property of the data, we decided to move the dictionary values to the ArrayData structure, making it natural for dictionary-encoded arrays to share the same DataType instance, regardless of the encoding details.

@xhochy
Copy link

xhochy commented Nov 15, 2019

There is also the 0.15.x series out now.

@randyzwitch
Copy link
Contributor Author

Thanks @xhochy, we’re evaluating if we can jump right to 0.15

@mikehinchey
Copy link
Contributor

Pymapd now uses pyarrow 0.15.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants