Access to Hadoop's HDFS via libhdfs and JNI
Pull request Compare This branch is even with forward:master.
Latest commit c2a720e Sep 13, 2011 @horaci horaci Updated package.json
Failed to load latest commit information.
demo added exists, mkdir, rm and append Sep 12, 2011
src added exists, mkdir, rm and append Sep 12, 2011
vendor hack in hdfs.h and building with libs Jun 27, 2011
.gitignore Initial rewrite Sep 7, 2011 fix error in example in README Sep 12, 2011
node-hdfs.js added exists, mkdir, rm and append Sep 12, 2011
wscript Remove some commented out stuff in the waf configure, don't set HADOO… Sep 12, 2011


This provides a JNI-implemented HDFS client allowing node.js applications to natively interoperate with the Hadoop FileSystem.

Example Use

There's a run script that will build + run the demo/demo.js sample.

var HDFS = require('node-hdfs');
var client = new HDFS({host:"default", port:0});

client.list("/tmp/path", function(err, files) {


At the moment it's still a little tricky. At the least you'll need to make sure libhdfs is built and installed in a path accessible by ldconfig (i.e. /usr/local/lib).


I (Paul) have had success with Cloudera's CDH distribution. Please see their instructions for installing from packages.


cd [hadoop_uncompressed_src_path]/src/c++/libhdfs
chmod +x ./configure

To build libhdfs in Mac OSX a few changes need to be done in the generated 'Makefile' by './configure':

Edit Makefile and search for a line that starts with 'CFLAGS = -g O2'

  • Remove the parameter "-m"
  • Add parameter "-framework JavaVM"

Search for a line that starts with 'DEFS = -DPACKAGE_NAME=\"libhdfs\" -DPACKAGE_TARNAME=\"libhdfs\"'

  • Remove the parameter "-Dsize_t=unsigned\ int"

Comment or remove include in 'hdfsJniHelper.c' :

Search for '#include <error.h>' and remove (or comment it out).

Then build

chmod +x ./install-sh
make install

Copy the libraries to a folder accessible to the compiler or add it to the env with dyld (man dyld for more information).

cd [hadoop_uncompressed_src_path]/src/c++/install/lib
cp libhdfs* /usr/local/lib