hfind is a find(1) implementation for Hadoop.
Download the latest version of the hfind tarball here.
The tarball contains two files, a shell script and a jar.
You first need to configure hfind to find your Hadoop installation, via the HFIND_OPTS variable:
export HFIND_OPTS="-Dhfind.hadoop.ugi='pierre\,pierre' -Dhfind.hadoop.namenode.url=hdfs://namenode.company.com:9000"
You can then simply run ./hfind from the exploded tarball directory:
[pierre@mouraf ~/downloads]$ ls metrics.hfind-1.0.0-SNAPSHOT.tar.gz [pierre@mouraf ~/downloads]$ tar zxvf metrics.hfind-1.0.0-SNAPSHOT.tar.gz hfind target/metrics.hfind-1.0.0-SNAPSHOT-jar-with-dependencies.jar [pierre@mouraf ~/downloads]$ export HFIND_OPTS="-Dhfind.hadoop.ugi='pierre\,pierre' -Dhfind.hadoop.namenode.url=hdfs://namenode.ning.com:9 000" [pierre@mouraf ~/downloads]$ ./hfind /user/pierre -type f -name "file*.dat" -o -type d -name a /user/pierre/testing/a /user/pierre/testing/a/b/fileb.dat /user/pierre/testing/a/c/filec.dat /user/pierre/testing/a/filea.dat /user/pierre/testing/a/fileaa.dat
[pierre@mouraf ~]$ hfind /user/pierre -type f -name "file*.dat" -o -type d -name a /pierre/testing/a /pierre/testing/a/b/fileb.dat [pierre@mouraf ~]$ hfind /user/pierre -mtime +3 -type d /user/pierre/2010/09 /user/pierre/2010/09/20 /user/pierre/2010/09/21
hfind follows as close as possible the POSIX specification documented here.
As of 2010, some options cannot be implemented (e.g. symbolic links).
-maxdepth/-d follows the BSD implementation:
-d Cause find to perform a depth-first traversal, i.e., directories are visited in post-order and all entries in a
directory will be acted on before the directory itself. By default, find visits directories in pre-order, i.e.,
before their contents. Note, the default is not a breadth-first traversal.
This option is equivalent to the -depth primary of IEEE Std 1003.1-2001 (``POSIX.1'').
-maxdepth n
Descend at most n directory levels below the command line arguments. If any -maxdepth
primary is specified, it applies to the entire expression even if it would not normally be evaluated.
-maxdepth 0 limits the whole search to the command line arguments.
See TODO below for work in progress.
A directory size is always reported as zero. Use -empty to find empty directories.
A directory mtime is the date of the latest modification of any file in the directory, going one level only. You can have more recent files in subdirectories:
[pierre@mouraf ~]$ find /tmp/a -ls 45404725 0 drwxr-xr-x 4 pierre wheel 136 Sep 23 17:10 /tmp/a 45404727 0 drwxr-xr-x 3 pierre wheel 102 Sep 23 17:11 /tmp/a/b 45404730 0 -rw-r--r-- 1 pierre wheel 0 Sep 23 17:11 /tmp/a/b/hello 45404726 0 -rw-r--r-- 1 pierre wheel 0 Sep 23 17:10 /tmp/a/hello
In the previous example, mtime of /tmp/a is 17:10 although, it contains /tmp/a/b/hello, modified later.
-
Respect parentheses
-
Implement -prune
-
Implement -exec and -ok
-
Implement -perm
-
Implement -depth and -mindepth
mvn install
By default, hfind bundles Hadoop version 0.20.2. One can override it via the hadoop.version parameter, e.g.
mvn clean install mvn -Dhadoop.version=0.20.2-cdh3u1 clean install mvn -Dhadoop.version=0.20.2-cdh3u2 clean install mvn -Dhadoop.version=0.20.2-cdh3u3 clean install mvn -Dhadoop.version=1.0.0 clean install mvn -Dhadoop.version=1.0.1 clean install mvn -Dhadoop.version=1.0.2 clean install
Copyright 2010-2012 Ning
Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.