Skip to content

Commit

Permalink
Add hadoop dependencies into ozone ufs connector
Browse files Browse the repository at this point in the history
### What changes are proposed in this pull request?

Tested in
[ozone-1.2.1](https://github.com/apache/ozone/releases/tag/ozone-1.2.1)
and ozone master branch(1.3.0-SNAPSHOT)

Mount ozone with the following exception.

```
java.lang.IllegalArgumentException: Unable to create an UnderFileSystem
instance for path: o3fs://testbuk.testvol/
at
alluxio.underfs.UnderFileSystem$Factory.create(UnderFileSystem.java:120)
at
alluxio.underfs.AbstractUfsManager.getOrAdd(AbstractUfsManager.java:136)
at
alluxio.underfs.AbstractUfsManager.lambda$addMount$2(AbstractUfsManager.java:203)
at
alluxio.underfs.UfsManager$UfsClient.acquireUfsResource(UfsManager.java:61)
at
alluxio.master.file.DefaultFileSystemMaster.prepareForMount(DefaultFileSystemMaster.java:3164)
at
alluxio.master.file.DefaultFileSystemMaster.mountInternal(DefaultFileSystemMaster.java:3327)
at
alluxio.master.file.DefaultFileSystemMaster.mountInternal(DefaultFileSystemMaster.java:3289)
at
alluxio.master.file.DefaultFileSystemMaster.mount(DefaultFileSystemMaster.java:3264)
at
alluxio.master.file.FileSystemMasterClientServiceHandler.lambda$mount$11(FileSystemMasterClientServiceHandler.java:285)
at alluxio.RpcUtils.callAndReturn(RpcUtils.java:123)
at alluxio.RpcUtils.call(RpcUtils.java:84)
at alluxio.RpcUtils.call(RpcUtils.java:59)
at
alluxio.master.file.FileSystemMasterClientServiceHandler.mount(FileSystemMasterClientServiceHandler.java:284)
at
alluxio.grpc.FileSystemMasterClientServiceGrpc$MethodHandlers.invoke(FileSystemMasterClientServiceGrpc.java:2700)
at
io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
at
io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at
io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at
io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at
alluxio.security.authentication.ClientIpAddressInjector$1.onHalfClose(ClientIpAddressInjector.java:57)
at
io.grpc.PartialForwardingServerCallListener.onHalfClose(PartialForwardingServerCallListener.java:35)
at
io.grpc.ForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:23)
at

io.grpc.ForwardingServerCallListener$SimpleForwardingServerCallListener.onHalfClose(ForwardingServerCallListener.java:40)
at
alluxio.security.authentication.AuthenticatedUserInjector$1.onHalfClose(AuthenticatedUserInjector.java:67)
at
io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:331)
at
io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:797)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at
io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Suppressed: java.lang.NoClassDefFoundError:
org/apache/hadoop/thirdparty/com/google/common/base/Preconditions
at
org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:430)
at
org.apache.hadoop.conf.Configuration$DeprecationDelta.<init>(Configuration.java:443)
at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:525)
at
alluxio.underfs.hdfs.HdfsUnderFileSystem.createConfiguration(HdfsUnderFileSystem.java:267)
at
alluxio.underfs.hdfs.HdfsUnderFileSystem.createInstance(HdfsUnderFileSystem.java:123)
at
alluxio.underfs.hdfs.HdfsUnderFileSystemFactory.create(HdfsUnderFileSystemFactory.java:42)
at
alluxio.underfs.UnderFileSystem$Factory.create(UnderFileSystem.java:105)
... 29 more
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.thirdparty.com.google.common.base.Preconditions
at java.base/java.lang.ClassLoader.findClass(ClassLoader.java:719)
at
alluxio.extensions.ExtensionsClassLoader$DefaultClassLoader.findClass(ExtensionsClassLoader.java:43)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
at
alluxio.extensions.ExtensionsClassLoader$DefaultClassLoader.loadClass(ExtensionsClassLoader.java:53)
at
alluxio.extensions.ExtensionsClassLoader.loadClass(ExtensionsClassLoader.java:84)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
... 36 more
```

pr-link: #16336
change-id: cid-75134b178900eee4e46afdbdac88ce4d23c3af68
  • Loading branch information
maobaolong committed Nov 7, 2022
1 parent a1e1e33 commit d0d2980
Showing 1 changed file with 0 additions and 10 deletions.
10 changes: 0 additions & 10 deletions underfs/ozone/pom.xml
Expand Up @@ -73,16 +73,6 @@
<groupId>org.alluxio</groupId>
<artifactId>alluxio-underfs-hdfs</artifactId>
<version>${project.version}</version>
<exclusions>
<exclusion>
<artifactId>hadoop-common</artifactId>
<groupId>org.apache.hadoop</groupId>
</exclusion>
<exclusion>
<artifactId>hadoop-hdfs-client</artifactId>
<groupId>org.apache.hadoop</groupId>
</exclusion>
</exclusions>
</dependency>
</dependencies>

Expand Down

0 comments on commit d0d2980

Please sign in to comment.