Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]The pod failed to mount pvc, and fuse reported the error "Failed to mount ufs path..." #4124

Closed
starZane opened this issue May 23, 2024 · 4 comments
Labels
bug Something isn't working

Comments

@starZane
Copy link

starZane commented May 23, 2024

What is your environment(Kubernetes version, Fluid version, etc.)
k8s version 1.25.4+
ubuntu 22.04
kernel 5.15.0-94-generic aarch64
fluid version 0.9.2
alluxio-dev version 2.9.1-gamma

Describe the bug
image
image

What you expect to happen:
I tried to use demo to test and encountered this error

dataset

apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
  name: demo
  namespace: fluid-system
spec:
  mounts:
  - mountPoint: s3://fluid/test
    name: fluid
    options:
      alluxio.underfs.s3.endpoint: http://xx.xxx.xxx.xxx:xxx
      alluxio.underfs.s3.disable.dns.buckets: "true"
      alluxio.underfs.s3.inherit.acl: "false"
    encryptOptions:
    - name: aws.accessKeyId
      valueFrom:
        secretKeyRef:
          name: mysecret
          key: aws.accessKeyId
    - name: aws.secretKey
      valueFrom:
        secretKeyRef:
          name: mysecret
          key: aws.secretKey

secret

Omit

runtime

apiVersion: data.fluid.io/v1alpha1
kind: AlluxioRuntime
metadata:
  name: demo
  namespace: fluid-system
spec:
  replicas: 1
  tieredstore:
    levels:
      - mediumtype: MEM
        path: /dev/shm
        quota: 20M
        high: "0.95"
        low: "0.7"

pod

apiVersion: v1
kind: Pod
metadata:
  name: demo-app
  namespace: fluid-system
spec:
  containers:
    - name: demo
      image: nginx:latest
      volumeMounts:
        - mountPath: /data
          name: demo
  volumes:
    - name: demo
      persistentVolumeClaim:
        claimName: demo

How to reproduce it

Additional Information

@starZane starZane added the bug Something isn't working label May 23, 2024
@cheyang
Copy link
Collaborator

cheyang commented May 24, 2024

@starZane Could you send the log of demo-fuse-fqtpk without secret? Btw, may I know the reason of using alluixo 2.9.1-gamma?

@starZane
Copy link
Author

starZane commented May 24, 2024

All log contents of demo-fuse-fqtpk. I also didn't find how to enable DEBUG level to get more logs:
Mounting --fuse-opts=kernel_cache,ro,attr_timeout=7200,entry_timeout=7200,allow_other to /runtime-mnt/alluxio/fluid-system/demo/alluxio-fuse...........................................................
[2024-05-23T09:39:44+0000]: Failed to mount ufs path --fuse-opts=kernel_cache,ro,attr_timeout=7200,entry_timeout=7200,allow_other to local mount point /runtime-mnt/alluxio/fluid-system/demo/alluxio-fuse.

I tried to use version 2.9.0 to create an aarch64 image, but an architecture problem was reported:
Exception in thread "main" java.lang.RuntimeException: Invalid property key ALLUXIO_CLIENT_HOSTNAME
at alluxio.conf.InstancedConfiguration.lookupRecursively(InstancedConfiguration.java:441)
at alluxio.conf.InstancedConfiguration.lookup(InstancedConfiguration.java:412)
at alluxio.conf.InstancedConfiguration.isResolvable(InstancedConfiguration.java:152)
at alluxio.conf.InstancedConfiguration.isSet(InstancedConfiguration.java:162)
at alluxio.conf.AlluxioConfiguration.getOrDefault(AlluxioConfiguration.java:65)
at alluxio.cli.GetConf.getConfImpl(GetConf.java:189)
at alluxio.cli.GetConf.getConf(GetConf.java:146)
at alluxio.cli.GetConf.main(GetConf.java:267)
Path /runtime-mnt/alluxio/fluid-system/demo/alluxio-fuse is not mounted
Starting AlluxioFuse process: mounting alluxio path "/" to local mount point "/runtime-mnt/alluxio/fluid-system/demo/alluxio-fuse"
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
2024-05-23 05:57:31,777 INFO AlluxioFuse - Alluxio version: 2.9.0-d5919d8d80ae7bfdd914ade30620d5ca14f3b67e
2024-05-23 05:57:32,113 INFO AlluxioFuse - Set fuse mount point options as kernel_cache,ro,attr_timeout=7200,entry_timeout=7200,allow_other from command line input
2024-05-23 05:57:32,199 INFO MetricsSystem - Starting sinks with config: {}.
2024-05-23 05:57:32,207 INFO MetricsHeartbeatContext - Created metrics heartbeat with ID app-311639718358342106. This ID will be used for identifying info from the client. It can be set manually through the alluxio.user.app.id property
2024-05-23 05:57:32,358 INFO NettyUtils - EPOLL_MODE is available
2024-05-23 05:57:32,829 INFO TieredIdentityFactory - Initialized tiered identity TieredIdentity(node=10.110.56.161, rack=null)
2024-05-23 05:57:32,873 ERROR AlluxioFuse - Failed to launch FUSE
java.lang.UnsatisfiedLinkError: /tmp/libjnifuse4366757128820846893.so: /tmp/libjnifuse4366757128820846893.so: cannot open shared object file: No such file or directory (Possible cause: can't load AMD 64-bit .so on a AARCH64-bit platform)
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
at java.lang.Runtime.load0(Runtime.java:782)
at java.lang.System.load(System.java:1100)
at alluxio.jnifuse.utils.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:231)
at alluxio.jnifuse.utils.NativeLibraryLoader.lambda$load$2(NativeLibraryLoader.java:126)
at alluxio.jnifuse.utils.NativeLibraryLoader.tryLoad(NativeLibraryLoader.java:84)
at alluxio.jnifuse.utils.NativeLibraryLoader.load(NativeLibraryLoader.java:126)
at alluxio.jnifuse.utils.NativeLibraryLoader.load3(NativeLibraryLoader.java:162)
at alluxio.jnifuse.utils.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:197)
at alluxio.jnifuse.LibFuse.loadLibrary(LibFuse.java:45)
at alluxio.fuse.AlluxioFuse.launchFuse(AlluxioFuse.java:160)
at alluxio.fuse.AlluxioFuse.main(AlluxioFuse.java:140)

So I tried to use new version to avoid this problem

@cheyang
Copy link
Collaborator

cheyang commented May 28, 2024

I checked the alluxio docker hub repo: https://hub.docker.com/r/alluxio/alluxio-dev/tags?page=&page_size=&ordering=&name=2.9. I didn't see alluxio support ARM64 officially. So I suggest you asking alluxio to support ARm64 firstly.

And 2.9.1 gamma is not official released version, I'm not sure if it can work fine.

@starZane
Copy link
Author

OK,thank you for your reply!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants