Mounting an Ozone FS with fuse-dfs or similar

[gigem]

How well does fuse-dfs work with Ozone? Are there any alternatives that work better? In some cases, we have non-S3/HFFS aware applications which need to check the presence of files and copy data using standard, Unix file access.

[kerneltime]

@gigem I have not heard of any one trying this out. Some of the semantics on the write path might not be quite what FUSE expects (there are caveats listed in fuse-dfs page and might be longer for Ozone). If you want to give it a try we can try to help with any issues you encounter.

[kerneltime]

There is an easier way, https://github.com/apache/ozone/blob/master/hadoop-ozone/ozonefs-hadoop3/pom.xml generates a shaded fat jar with all the dependencies included. You can also use https://github.com/apache/ozone/blob/master/hadoop-ozone/ozonefs-hadoop3-client/pom.xml#L30 which generates dependencies based on hadoop-client.jar

You should only need to place one of these jars in the class path for the ofs:// URLs to work. Do you have the config files in place for Ozone?

[gigem]

I will try the fat jars tomorrow. Yes, that should be much simper. I'm running fuse_dfs on a client system itself run Ozone. It looks like when I copied what I thought I needed from the bukld system, I forgot to copy the config file. I will also correct that tomorrow. Thanks for you help.

[gigem]

@kerneltime, still no joy. With CLASSPATH set like this:
export CLASSPATH=/home/dengel/src/ozone/hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3-1.4.0-SNAPSHOT.jar
I get the following when running fuse_dfs to mount:
FUSE library version: 2.9.9 nullpath_ok: 0 nopath: 0 utime_omit_ok: 0 unique: 2, opcode: INIT (26), nodeid: 0, insize: 56, pid: 0 INIT: 7.34 flags=0x33fffffb max_readahead=0x00020000 could not find method getRootCauseMessage from class (null) with signature (Ljava/lang/Throwable;)Ljava/lang/String; could not find method getStackTrace from class (null) with signature (Ljava/lang/Throwable;)Ljava/lang/String; FileSystem: loadFileSystems failed error: (unable to get root cause for java.lang.NoClassDefFoundError) (unable to get stack trace for java.lang.NoClassDefFoundError) getJNIEnv: getGlobalJNIEnv failed Unable to determine the configured value for hadoop.fuse.timer.period.ERROR /home/dengel/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_init.c:134 FATAL: dfs_init: fuseConnectInit failed with error -22! ERROR /home/dengel/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_init.c:34 LD_LIBRARY_PATH=NULL ERROR /home/dengel/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_init.c:35 CLASSPATH=/home/dengel/src/ozone/hadoop-ozone/ozonefs-hadoop3/target/ozone-filesystem-hadoop3-1.4.0-SNAPSHOT.jar INFO /home/dengel/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_options.c:167 Adding FUSE arg mnt/ozone INFO /home/dengel/hadoop/hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/fuse_init.c:98 Mounting with options: [ protected=(NULL), nn_uri=ofs://kbtest01/hivemr3/warehouse, nn_port=0, debug=1, read_only=0, initchecks=0, no_permissions=0, usetrash=0, entry_timeout=60, attribute_timeout=60, rdbuffer_size=10485760, direct_io=0, max_background=0 ]
`

[kerneltime]

getGlobalJNIEnv failed Unable to determine the configured value for hadoop.fuse.timer.period.ERROR Seems like a config issue?

[gigem]

I had other things to attend to so couldn't look at this again until today.

The original problem using the full CLASSPATH appears to have been due to a Java difference between the server and client systems. I initially simply rsynced the compiled Ozone directory from the server to the client. I rebuilt Ozone on the client and that now works.

Even with the rebuild on the client, using the shaded jars still does not work. There seems to be some class(es) missing that prevents the JNI environment from initializing. I will try a few more things but with the full CLASSPATH working, I likely won't spend much more time on it.