Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update PyArrow to 0.16.0 #9119

Merged
merged 1 commit into from Mar 8, 2020
Merged

Conversation

robdiciuccio
Copy link
Member

@robdiciuccio robdiciuccio commented Feb 11, 2020

CATEGORY

Choose one

  • Bug Fix
  • Enhancement (new features, refinement)
  • Refactor
  • Add tests
  • Build / Development Environment
  • Documentation

SUMMARY

Update PyArrow from 0.15.1 to 0.16.0. Includes a number of fixes and enhancements, including improvements to Pandas conversion with nested data types.

Complete CHANGELOG:
https://github.com/apache/arrow/blob/master/CHANGELOG.md

Includes a fix for #8843

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

  • CI tests pass
  • Queries run in SQL Lab succeed

ADDITIONAL INFORMATION

REVIEWERS

@john-bodley @serenajiang @dpgaspar @nytai @villebro

@john-bodley
Copy link
Member

@robdiciuccio do you know if the bug fixes will resolve the issue @serenajiang reported in #9096 (comment).

@codecov-io
Copy link

codecov-io commented Feb 11, 2020

Codecov Report

Merging #9119 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #9119   +/-   ##
=======================================
  Coverage   58.91%   58.91%           
=======================================
  Files         372      372           
  Lines       11996    11996           
  Branches     2937     2937           
=======================================
  Hits         7068     7068           
  Misses       4750     4750           
  Partials      178      178

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f73f8a...7e4527f. Read the comment docs.

@robdiciuccio
Copy link
Member Author

@john-bodley this does not appear to handle the issue with lists containing mixed int and string values. We'll have to handle that explicitly with additional serialization.

@robdiciuccio
Copy link
Member Author

Seeing a potential issue with this new version raising OSError: Invalid flatbuffers message.. Investigating.

@robdiciuccio
Copy link
Member Author

Unable to reproduce the OSError after rebasing. Ready for review.

Copy link
Member

@willbarrett willbarrett left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@villebro villebro merged commit 8cabea5 into apache:master Mar 8, 2020
@willbarrett willbarrett deleted the rd/pyarrow-0.16.0 branch March 9, 2020 16:11
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.36.0 labels Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/XS 🚢 0.36.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unable to build on Alpine Linux with PyArrow 0.15.1
6 participants