[Python] Allow config hadoop_bin in pyarrow hdfs.py

[asfimport]

Currently, the hadoop_bin is either from HADOOP_HOME or the hadoop command. 

https://github.com/apache/arrow/blob/master/python/pyarrow/hdfs.py#L130

However, in some of environment setup, hadoop_bin could be some other location. Can we do something like 

 

if 'HADOOP_BIN' in os.environ:
    hadoop_bin = os.environ['HADOOP_BIN']
elif 'HADOOP_HOME' in os.environ:
    hadoop_bin = '{0}/bin/hadoop'.format(os.environ['HADOOP_HOME'])
else:
    hadoop_bin = 'hadoop'

 

 

Reporter: Wenbo Zhao

PRs and other links:

• GitHub Pull Request #2748

_{Note: This issue was originally created as ARROW-3503. Please see the migration documentation for further details.}

[asfimport]

Wes McKinney / @wesm:
Can you update your PR so we can merge this?

[asfimport]

Antoine Pitrou / @pitrou:
@ianmcook What do you think about this suggestion?

[asfimport]

Ian Cook / @ianmcook:
I think it is fairly uncommon for HADOOP_BIN to be in some other location.

[~wbzhao] are you interested in rebasing and updating the PR (#2748 to resolve conflicts that have been introduced by other commits since the PR was originally created? If not I think we could close this as "won't fix". Thanks.

[asfimport]

Antoine Pitrou / @pitrou:
Closing for now. If you want this to happen, please submit a PR and we can reopen this issue!