Latest Cloudera hadoop bundles do not include org.apache.hadoop.fs in their hadoop-core jar file

[michael-robbins]

Hey,

Trying to compile this with a later version of hadoop:

mvn -Dhadoop.version=2.0.0-mr1-cdh4.3.1 clean install

But when doing this I get a bunch of compile errors that it cannot find the package org.apache.hadoop.fs

[ERROR] /path/to/git/checkout/hfind/src/main/java/com/ning/hfind/FileStatusAttributes.java:[19,27] error: package org.apache.hadoop.fs does not exist

Any idea about this issue? I know it's old code and everything has probably changed since you released this. Maybe there is a native -find function now?

[pierre]

The APIs have changed with the latest releases, but it shouldn't be too hard to update the code to the latest version.

Regarding the native function, check https://issues.apache.org/jira/browse/HADOOP-8989.

[pierre]

Could you try the following patch?

diff --git a/pom.xml b/pom.xml
index 5c5a9b2..c5dbe30 100644
--- a/pom.xml
+++ b/pom.xml
@@ -4,7 +4,7 @@
     <groupId>com.ning</groupId>
     <artifactId>metrics.hfind</artifactId>
     <packaging>jar</packaging>
-    <version>1.0.2-SNAPSHOT</version>
+    <version>2.0.0-SNAPSHOT</version>
     <name>hfind</name>
     <description>A find implementation for Hadoop</description>
     <url>http://github.com/pierre/hfind</url>
@@ -28,7 +28,7 @@
     </repositories>
     <properties>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
-        <hadoop.version>0.20.2</hadoop.version>
+        <hadoop.version>2.2.0</hadoop.version>
     </properties>
     <dependencies>
         <dependency>
@@ -58,7 +58,7 @@
         </dependency>
         <dependency>
             <groupId>org.apache.hadoop</groupId>
-            <artifactId>hadoop-core</artifactId>
+            <artifactId>hadoop-client</artifactId>
             <version>${hadoop.version}</version>
             <exclusions>
                 <exclusion>
@@ -91,6 +91,11 @@
             <classifier>jdk15</classifier>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>oro</groupId>
+            <artifactId>oro</artifactId>
+            <version>2.0.8</version>
+        </dependency>
     </dependencies>
     <build>
         <plugins>

[michael-robbins]

It works! Cheers mate!

$> mvn -Dhadoop.version=2.0.0-mr1-cdh4.3.1 clean install
....
Build Output
....
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3:18.502s
[INFO] Finished at: Wed Dec 11 11:26:57 EST 2013
[INFO] Final Memory: 29M/72M
[INFO] ------------------------------------------------------------------------