Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blobfuse2 fstab not working #921

Closed
AGELadviseurs opened this issue Sep 28, 2022 · 15 comments
Closed

Blobfuse2 fstab not working #921

AGELadviseurs opened this issue Sep 28, 2022 · 15 comments

Comments

@AGELadviseurs
Copy link

Which version of the blobfuse was used?

2.0.0-preview.3

Which OS (please include version) are you using?

Ubuntu Server 20.04.5 LTS

What problem was encountered?

One-time mounting works fine. Can't get persistent mount working with fstab. No problems with blobfuse1.

Have you found a mitigation/solution?

No

By default, blobfuse logs errors to syslog. If this is relevant, is there anything in the syslog that might be helpful?

No

If relevant, please share your mount command.

In fstab I have the following entry:
blobfuse2 /mnt/blobcontainer fuse defaults,_netdev,--config-file=/home/user/blobfuse_config.yaml 0 0

sudo mount -a results in:
Error: unknown command "/mnt/blobcontainer" for "blobfuse2"

Adding "mount" after blobfuse2 in fstab and then trying sudo mount -a results in:
mount: /etc/fstab: parse error at line 5 -- ignored

Nothing related being output to varlog.

@gapra-msft
Copy link
Member

Hi @AGELadviseurs,

This may be due to the fact that the mount command for v2 is blobfuse2 mount /mnt/blobcontainer --config-file=/home/user/blobfuse_config.yaml Note the addition of the word mount. Could you try adding that please and let us know if the mount works?

@gapra-msft gapra-msft self-assigned this Sep 28, 2022
@AGELadviseurs
Copy link
Author

Hi @AGELadviseurs,

This may be due to the fact that the mount command for v2 is blobfuse2 mount /mnt/blobcontainer --config-file=/home/user/blobfuse_config.yaml Note the addition of the word mount. Could you try adding that please and let us know if the mount works?

Hi,

I tried that aswell (check the last paragraph in my first post) but it is not working. It says “parse error”.

@vibhansa-msft vibhansa-msft added this to the 2.0.0-preview.4 milestone Sep 29, 2022
@vibhansa-msft
Copy link
Member

@AGELadviseurs : we are able to reproduce this locally and it appears that having "mount" as a subcommand in blobfuse2 is causing this parsing error. This is because fstab expects first keyword to be file-system and second to be mount point while in case of blobfuse2 second keyword in command is "mount" subcommand. We are checking feasibility of getting around this problem. For now, there is one possible solution to this, if you are blocked due to this error. You can create a shell script that has the mount command written inside and fstab can just run that script.
For e.g.
/etc/fstab:
blobfuse2.sh /mnt/blobcontainer fuse defaults,_netdev,--config-file=/home/user/blobfuse_config.yaml 0 0

blobfuse2.sh
blobfuse2 mount /mnt/blobcontainer --config-file=/home/user/blobfuse_config.yaml

@AGELadviseurs
Copy link
Author

@vibhansa-msft: thanks, that workaround works for now.

@vibhansa-msft
Copy link
Member

Fixed has been merged in main branch, next release will have it.

@adgbu
Copy link

adgbu commented Dec 12, 2022

What was the fix?
What is the correct syntax to put in /etc/fstab now (blobfuse2 version 2.0.1)?

@vibhansa-msft
Copy link
Member

blobfuse2 /mnt/blob_mnt fuse defaults,netdev,allow_other,--tmp-path=/usr/tempcache,--config-file=/home/myuser/myblob.cfg 0 0
However, this will mount with root user.

@adgbu
Copy link

adgbu commented Dec 12, 2022

For some reason that does not work for me. Not sure what to look for in the logs.
Also please add "nofail" to the list of options, otherwise the system fails to boot at all when the mount fails.
I have the tmp-path set in the yaml file, so I omit it from the line in fstab.
I know the config is good, because I can mount it manually and I also have it working automatically at boot via systemd.

@vibhansa-msft
Copy link
Member

If your CLI command is working fine for mount, then you can put that up in shell script and from /etc/fstab you can just launch the shell script itself. This is one workaround if you are stuck on /etc/fstab.

@vibhansa-msft
Copy link
Member

Those customer facing this issue with fstab here are the updates:

  1. Earlier version of blobfuse were not fstab compliant because "mount" subcommand is required in blobfuse2 cli and fstab does not support that feature. If you are facing this issue you need to upgrade to our GA version.
  2. If the issue persists (it has been fixed in GA version by making mount as default sub command) workaround is to put your mount command in a shell script and run that script from fstab. Also, report it as a new github issue so that we can investigate it from our end.

@cl33n
Copy link

cl33n commented Apr 3, 2023

Hi all -
Reviving this thread because I'm still failing to get blobfuse2 to mount via fstab or systemd. It mounts fine via azure cli, and my current workaround is to use an init.d script at rc3 instead. Not sure what I'm doing wrong:
Linux 3.10.0-1160.88.1.el7.x86_64
blobfuse2 version 2.0.3

fstab line:
blobfuse2 /home/username/mountdir fuse allow_other,--tmp-path=/home/username/tmpmount,--config-file=/usr/bin/azure-storage-fuse/setup/baseConfig.yaml 0 0

baseConfig.yaml:
Refer ./setup/baseConfig.yaml for full set of config parameters

allow-other: true

logging:
type: syslog
level: log_debug

components:

  • libfuse
  • stream
  • attr_cache
  • azstorage

libfuse:
attribute-expiration-sec: 120
entry-expiration-sec: 120
negative-entry-expiration-sec: 240

stream:
block-size-mb: 8
blocks-per-file: 3
cache-size-mb: 1024

attr_cache:
timeout-sec: 7200

azstorage:
type: block
account-name: redacted
container: redacted-test
endpoint: https://redacted.blob.core.windows.net
mode: key
account-key: redacted

This boots into emergency mode because the mount fails to load. My initial suspicion was that this failed because it tried to mount before the network was loaded, causing it to timeout during authentication. Setting the second 0 to 2 (to skip filesystem check at boot) fails too, which made me think something else isn't working.

I can share the systemd info as well if requested. Any thoughts? Thanks!

@AGELadviseurs
Copy link
Author

Hi all - Reviving this thread because I'm still failing to get blobfuse2 to mount via fstab or systemd. It mounts fine via azure cli, and my current workaround is to use an init.d script at rc3 instead. Not sure what I'm doing wrong: Linux 3.10.0-1160.88.1.el7.x86_64 blobfuse2 version 2.0.3

fstab line: blobfuse2 /home/username/mountdir fuse allow_other,--tmp-path=/home/username/tmpmount,--config-file=/usr/bin/azure-storage-fuse/setup/baseConfig.yaml 0 0

baseConfig.yaml: Refer ./setup/baseConfig.yaml for full set of config parameters

allow-other: true

logging: type: syslog level: log_debug

components:

  • libfuse
  • stream
  • attr_cache
  • azstorage

libfuse: attribute-expiration-sec: 120 entry-expiration-sec: 120 negative-entry-expiration-sec: 240

stream: block-size-mb: 8 blocks-per-file: 3 cache-size-mb: 1024

attr_cache: timeout-sec: 7200

azstorage: type: block account-name: redacted container: redacted-test endpoint: https://redacted.blob.core.windows.net mode: key account-key: redacted

This boots into emergency mode because the mount fails to load. My initial suspicion was that this failed because it tried to mount before the network was loaded, causing it to timeout during authentication. Setting the second 0 to 2 (to skip filesystem check at boot) fails too, which made me think something else isn't working.

I can share the systemd info as well if requested. Any thoughts? Thanks!

Hi,

I don't have any problems mounting through fstab. I see some differences though. First of all, I use the following line in fstab:

blobfuse2 /home/username/mountdir fuse defaults,_netdev,--config-file=/home/username/config.yaml,allow_other 0 0

I have the path for temp location inside the yaml file:

file_cache:
path: /tmp/blobfuse2tmp
timeout-sec: 120

@adgbu
Copy link

adgbu commented Apr 3, 2023

With blobfuse 1.4 I used fstab, but I never got it to work with blobfuse2. I have not tried since Dec 2022 though.

For what it is worth, below is my working unit file for systemd. With fstab the mount point is created automatically. With this systemd service solution, the mount point e.g. /mnt/blobs must already exist and be empty. The file_cache path specified in the yaml file must also exist and be empty. Hope this workaround helps someone.

[Unit]
Description=Mount an Azure blob container using blobfuse2 (FUSE).
Requires=network-online.target
After=network-online.target

[Service]
Type=forking
User=pi
Group=pi
ExecStart=blobfuse2 /mnt/blobs --config-file=/home/pi/blobfuse2-config.yaml
ExecStop=blobfuse2 unmount /mnt/blobs
Restart=on-failure
RestartSec=30

[Install]
WantedBy=multi-user.target

@cl33n
Copy link

cl33n commented Apr 4, 2023

Hi all - Reviving this thread because I'm still failing to get blobfuse2 to mount via fstab or systemd. It mounts fine via azure cli, and my current workaround is to use an init.d script at rc3 instead. Not sure what I'm doing wrong: Linux 3.10.0-1160.88.1.el7.x86_64 blobfuse2 version 2.0.3
fstab line: blobfuse2 /home/username/mountdir fuse allow_other,--tmp-path=/home/username/tmpmount,--config-file=/usr/bin/azure-storage-fuse/setup/baseConfig.yaml 0 0
baseConfig.yaml: Refer ./setup/baseConfig.yaml for full set of config parameters
allow-other: true
logging: type: syslog level: log_debug
components:

  • libfuse
  • stream
  • attr_cache
  • azstorage

libfuse: attribute-expiration-sec: 120 entry-expiration-sec: 120 negative-entry-expiration-sec: 240
stream: block-size-mb: 8 blocks-per-file: 3 cache-size-mb: 1024
attr_cache: timeout-sec: 7200
azstorage: type: block account-name: redacted container: redacted-test endpoint: https://redacted.blob.core.windows.net mode: key account-key: redacted
This boots into emergency mode because the mount fails to load. My initial suspicion was that this failed because it tried to mount before the network was loaded, causing it to timeout during authentication. Setting the second 0 to 2 (to skip filesystem check at boot) fails too, which made me think something else isn't working.
I can share the systemd info as well if requested. Any thoughts? Thanks!

Hi,

I don't have any problems mounting through fstab. I see some differences though. First of all, I use the following line in fstab:

blobfuse2 /home/username/mountdir fuse defaults,_netdev,--config-file=/home/username/config.yaml,allow_other 0 0

I have the path for temp location inside the yaml file:

file_cache: path: /tmp/blobfuse2tmp timeout-sec: 120

This worked for me, thanks so much.

@vibhansa-msft
Copy link
Member

_netdev : is the option that waits for the network device to come up. This might be the reason it was failing earlier. File-cache path provided in script or fstab shall not make any difference in the mount flow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants