Read hadoop configuration files (core-site.xml and hdfs-site.xml) in HdfsResource

[kaspersorensen]

Suggested fix for METAMODEL-219.

I did a few minor/additional changes too:

• Moved the HDFS outputstream and inputstream classes to separate files (HdfsResource was getting too big IMO). The classes are defined as default (non-public) scoped, so not visible outside the package.
• Changed the FileHelper to support Java 7 AutoCloseable instead of distinguishing between Closeable, Connection, Statement, ResultSet etc.

[LosD]

Missing EOL

[LosD]

Isn't it dangerous reuse existing Hadoop/Yarn environment variables? They may be set globally without being intended for use with whatever is using MetaModel, and they will override the configured fs.defaultFS, as far as I read the [JavaDocs](https://hadoop.apache.org/docs/r2.4.1/api/org/apache/hadoop/conf/Configuration.html#addResource%28java.io.InputStream, java.lang.String%29).

[kaspersorensen]

Fixed the EOLs.

That's a very good question actually... You're right that it would override fs.defaultFS, and initially that was also my intention. I see that HADOOP_CONF_DIR and YARN_CONF_DIR has become standards used by libraries such as Apache Spark and others. I was thinking it would be nice to reuse that standard so that configuration in MetaModel is as easy as possible... BUT of course you're right that it might be that the environment variable is unintentionally set up to configure a different hadoop installation. Hmm then I guess we shouldn't check for environment variables or system properties, but rather only use the constructor argument as a way of configuring it. Then applications that are using MetaModel have to do their own environment variable checking, if it's applicable in their situations.

[kaspersorensen]

OK as you can now see from the latest commit (0e0a9fb) I have added a system property metamodel.hadoop.use_hadoop_conf_dir which can be set to "true" in order to enable the HADOOP_CONF_DIR and YARN_CONF_DIR checks. I guess that disables it for anyone that's not explicitly enabling it.

[LosD]

Good solution. 👍, then :)