Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FutureWarning when importing pyarrow.HadoopFileSystem #600

Closed
tlinhart opened this issue May 13, 2021 · 3 comments · Fixed by #615
Closed

FutureWarning when importing pyarrow.HadoopFileSystem #600

tlinhart opened this issue May 13, 2021 · 3 comments · Fixed by #615
Labels
needs:testing Needs testing to reproduce new-contributor-friendly

Comments

@tlinhart
Copy link
Contributor

I'm getting this warning when using papermill (as a library from Python script):

WARNING:papermill:/home/pasmen/xxx/venv/lib/python3.8/site-packages/papermill/iorw.py:50: FutureWarning: pyarrow.HadoopFileSystem is deprecated as of 2.0.0, please use pyarrow.fs.HadoopFileSystem instead.
  from pyarrow import HadoopFileSystem

I'm using Python 3.8 with papermill 2.3.3 and pyarrow 4.0.0 (installed implicitly as a papermill dependency).

The fix should be straightforward, I might try to prepare PR if interested.
@willingc
Copy link
Member

willingc commented Jun 6, 2021

Thanks @tlinhart for the report.

@willingc willingc added the needs:testing Needs testing to reproduce label Jun 6, 2021
@MSeal
Copy link
Member

MSeal commented Jun 6, 2021

@tlinhart a PR to fix would be wonderful if you do have time. Always happy to get the future warnings resolved early.

@abhimanyu-bitsgoa abhimanyu-bitsgoa mentioned this issue Jun 7, 2021
Merged
@abhimanyu-bitsgoa
Copy link
Contributor

abhimanyu-bitsgoa commented Jun 7, 2021 •
edited

Hi!
I just created a pull request to address this issue. Needed to call a different import as shown here.
Kindly review. @willingc @MSeal

This was referenced Dec 2, 2021
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs:testing Needs testing to reproduce new-contributor-friendly
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Removing deprecated pyarrow.hdfs.connect call from iorw.py abhimanyu-bitsgoa/papermill
4 participants
@MSeal @willingc @tlinhart @abhimanyu-bitsgoa