Go bindings for libhdfs, for manipulating files on Hadoop distributed file system.
hdfs.Fs: file system handle
hdfs.File: file handle
hdfs.FileInfo: file metadata structure, represented within Go
- HDFS: c bindings for libhdfs, java binary packages
- HDFS: configured cluster
Tips for building libhdfs on OS X
Based on hadoop-1.0.4.
chmod +x src/c++/libhdfs/install-sh
ant -Dcompile.c++=true -Dlibhdfs=true compile-c++-libhdfsto build libhdfs.
- Upon successful building, libraries have been installed in
build/c++-build/Mac_OS_X-x86_64-64/libhdfsis very helpful for later-on re-compilation.
it is ok the build ends up with installation errors if you can already find compiled libs in
- change install_name for libhdfs:
install_name_tool -id /usr/lib/java/libhdfs.0.0.0.dylib libhdfs.0.0.0.dylib
Tips for building libhadoop on OS X
Based on hadoop-1.0.1; libhadoop would be loaded by
util.NativeCodeLoader when accessing local file system.
- java: change
-framework JavaVMin both
libz: apply patch to
elif test ! -z "`which otool | grep -v 'no otool'`"; then ac_cv_libname_$1=\"`otool -L conftest | grep $1 | sed -e 's/^[ ]*//' -e 's/ .*//' -e 's/.*\/\(.*\)$/\1/'`\";
elif test ! -z "`which otool | grep -v 'no otool'`"; then ac_cv_libname_z=\"`otool -L conftest | grep z | sed -e 's/^ *//' -e 's/ .*//' -e 's/.*\/\(.*\)$/\1/'`\";
apply patch to source code
- put the compiled library
libhadoop.1.0.0.dyliband its symbolic links in
/usr/lib/java, which is one of the default element of
- change install_name for libhadoop:
sudo install_name_tool -id /usr/lib/java/libhadoop.1.dylib /usr/lib/java/libhadoop.1.0.0.dylib
.jarfrom hadoop in
mktest.shfor details, or you can modify it to accommodate your environment.
libjvmare declared in
You don't have to do this on OS X. You can always use
install_name_toolto set or change a library's install name, also jvm on OS X is a system framework, so that it is not necessory to add jvm's path, while the only thing in step 3 is providing
hdfs.hheader path for #cgo.
correct the #cgo header in
hdfs.go, according to your enviornment.
- After the preparation, correct the constants in
Currently connecting to local file system is not handled correctly. SoIt is okay now to access to local file system.
Connect("", 0)would lead to error.
errnoin libhdfs is not handled precisely. For example,
invokeMethod()would probably sets
errnoto 2 in a lot of routines.