just build with plain 'mvn install'

[marcelmfs]

... and the result is the same as stock binary hfind jar:

Error crawling HDFS: Server IPC version 9 cannot communicate with client version 3

mvn install downloaded default libraries for the building process.

My hadoop version is 2.5.0-cdh5.3.3

Thanks

[pierre]

The stock jar is compatible with Hadoop 2.2.0 only. Could you try changing the version in the pom.xml?

[marcelmfs]

I changed the version to 2.5.0 on hadoop.version xml section. And now:

$ ./hfind / -type f Exception in thread "main" java.lang.VerifyError: (class: com/google/common/collect/Interners, method: newWeakInterner signature: ()Lcom/google/common/collect/Interner;) Incompatible argument to function at org.apache.hadoop.util.StringInterner.<clinit>(StringInterner.java:48) at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2303) at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2195) at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2112) at org.apache.hadoop.conf.Configuration.set(Configuration.java:989) at org.apache.hadoop.conf.Configuration.set(Configuration.java:961) at com.ning.hfind.config.HdfsConfig.configureHDFSAccess(HdfsConfig.java:20) at com.ning.hfind.config.HdfsConfig.<init>(HdfsConfig.java:12) at com.ning.hfind.HdfsAccess.<init>(HdfsAccess.java:27) at com.ning.hfind.HdfsAccess.get(HdfsAccess.java:20) at com.ning.hfind.primary.Expression.run(Expression.java:94) at com.ning.hfind.Find.main(Find.java:151)

[pierre]

Could you post the output of mvn dependency:tree? It looks like google-collections and Guava are both in the classpath.

[marcelmfs]

https://gist.github.com/djeps/76f4536a6b3cb962601c

[pierre]

Thanks!

Here is the issue:

[INFO] +- com.google.collections:google-collections:jar:1.0-rc2:compile
...
[INFO] +- org.apache.hadoop:hadoop-client:jar:2.5.0:compile
[INFO] |  +- org.apache.hadoop:hadoop-common:jar:2.5.0:compile
[INFO] |  |  +- com.google.guava:guava:jar:11.0.2:compile

Could you replace the google-collections dependency with Guava in your pom.xml? See b1e4a1b.

[marcelmfs]

$ ./hfind /users -type d
log4j:WARN No appenders could be found for logger (org.apache.hadoop.conf.Configuration.deprecation).
log4j:WARN Please initialize the log4j system properly.
Error crawling HDFS: No FileSystem for scheme: hdfs
$ hdfs dfs -ls -R hdfs://localhost:54310/
15/09/10 13:51:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
drwxr-xr-x   - username supergroup          0 2015-09-02 23:08 hdfs://localhost:54310/users
drwxr-xr-x   - username supergroup          0 2015-09-03 15:47 hdfs://localhost:54310/users/username
$ hdfs dfs -ls
$
$ egrep -v '^$|^#' hfind
export HFIND_OPTS="-Dhfind.hadoop.ugi='username\,supergroup' -Dhfind.hadoop.namenode.url=hdfs://localhost:54310 -Xms512m -Xmx3072m"
HFIND_JAR=${HFIND_JAR:=$PWD/target/metrics.hfind-*-jar-with-dependencies.jar}
java $HFIND_OPTS -jar $HFIND_JAR $@
$

[pierre]

Could you try adding hadoop-hdfs as a dependency?

<dependency>
    <groupId>org.apache.hadoop</groupId>
    <artifactId>hadoop-hdfs</artifactId>
    <version>${hadoop.version}</version>
</dependency>

See http://stackoverflow.com/questions/26958865/no-filesystem-for-scheme-hdfs/28135140#28135140.