[BEAM-8368] [BEAM-8392] Update pyarrow to the latest version

[aaltay]

Update pyarrow to the latest version.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---		---	---
Java
Python		---			---	---
XLang	---	---	---		---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[aaltay]

R: @TheNeuralBit

[TheNeuralBit]

Looks like maybe there was an API change in pyarrow.parquet. I'll take a look

[TheNeuralBit]

Well it's a pretty simple fix, just need to change this line to pa.Table.from_arrays([orig.column('name')], names=['name'])

[TheNeuralBit]

Wrote a commit for this and put up a PR to merge into this branch

[aaltay]

Thank you @TheNeuralBit!

[aaltay]

Well it's a pretty simple fix, just need to change this line to pa.Table.from_arrays([orig.column('name')], names=['name'])

A bit of a side question. It seems like minor versions could be breaking with pyarrow. Should we pin this library, instead of using an upper bound?

[aaltay]

Run Python PreCommit

[aaltay]

Run Python PreCommit

[TheNeuralBit]

A bit of a side question. It seems like minor versions could be breaking with pyarrow. Should we pin this library, instead of using an upper bound?

They're not supposed to be breaking, in this case I think the breaking change came from 0.14 to 0.15, rather than 0.15.0 to 0.15.1.