borg mount broken when using brew on macOS

[sunknudsen]

Have you checked borgbackup docs, FAQ, and open Github issues?

Yes

Is this a BUG / ISSUE report or a QUESTION?

BUG

System information. For client/server mode post info for both machines.

Your borg version (borg -V).

borg 1.1.14

Operating system (distribution) and version.

macOS Catalina 10.15.7

Full borg commandline that lead to the problem (leave away excludes and passwords)

borg mount ...

Describe the problem you're observing.

$ fuse mount ...
borg mount not available: loading FUSE support failed [ImportError: No module named 'llfuse']

Can you reproduce the problem? If so, describe how. If not, describe troubleshooting steps you took before opening the issue.

I believe there is an issue with recent versions of FUSE for macOS.

The above issue is present when https://github.com/osxfuse/osxfuse/releases/tag/osxfuse-3.11.2 is installed, but not when https://github.com/osxfuse/osxfuse/releases/tag/osxfuse-3.10.4 is installed.

[sunknudsen]

Just installed osxfuse-3.10.4 vs osxfuse-3.11.2 and I'm getting the same issue... Puzzled... osxfuse-3.10.4 works fine on Borg 1.1.14 on another computer.

[ThomasWaldmann]

You need to have llfuse installed also. Maybe there is a change in brew relating to that, they have some issue with fuse there - talk to them in case you use borg from brew.

[sunknudsen]

Thanks @ThomasWaldmann. Investigating...

[sunknudsen]

@ThomasWaldmann I believe I found the issue... do you believe the right decision has been made at the brew-level to remove llfuse resource?

See Homebrew/homebrew-core@89785cf#diff-4a25217474a5eb61d0776ab4cabc43b42689bc7b3efaaed400f799631dcec71f

My gut feeling is the Borg macOS documentation might have to be updated to account for this change.

[sunknudsen]

Btw, installing FUSE for macOS using brew cask install osxfuse is also broken.

[ThomasWaldmann]

Considering that macOS brew users work on a closed-sourced non-free operating system anyway, i can't really follow why they have an issue with FUSE for macOS and choose to cause issues for all users using functionality depending on that. But as I said, that is not a borg issue, but a brew policy issue.

[fxcoudert]

For the record: the Homebrew developers' point of view is that we try to depend only on things we can distribute ourselves, and the base system. We do not distribute osxfuse (since it is closed source), and therefore cannot enable configurations for other things that depend on it. In the past, we made an exception for Fuse explicitly, which we have decided to remove: it was causing support issues on our side, as well.

[fxcoudert]

That said, the fact that something is not in homebrew-core does not prevent to distribute it through Homebrew: homebrew is extensible and anyone can provide their own tap, under their own control and their own rules: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

[sunknudsen]

@fxcoudert Thanks for taking the time to share Homebrew's thought process with us. I understand why you guys removed FUSE from Homebrew... the question is how can we get borg mount working again. Any help would be much appreciated. I tried installing llfuse using pip3 install --user llfuse (having installed pkg-config first using brew install pkg-config) and same issue. I don't know much about Python... spent 2 hours trying to figure this out. I'm sure many other macOS users will benefit from this thread.

[fxcoudert]

One way is to take the content of the formula before Homebrew/homebrew-core@89785cf , meaning this: https://github.com/Homebrew/homebrew-core/blob/68158c398ff8076d7027c4525b78edcf31d1a664/Formula/borgbackup.rb

and host that in your own tap: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap

[sunknudsen]

@fxcoudert Have a moment to share how Homebrew installs llfuse from source and where the python library is located on macOS once installed?

The following lines are pretty abstracted... I would like to collaborate with Borg to update the documentation so users who wish to use borg mount on macOS can install dependencies manually.

Homebrew/homebrew-core@89785cf#diff-4a25217474a5eb61d0776ab4cabc43b42689bc7b3efaaed400f799631dcec71fL33-L37

[fxcoudert]

@sunknudsen run the installation with brew install -v and it will show you what is done. It's creating a virtualenv and simply installing the packages from source with pip

[sunknudsen]

Thanks @fxcoudert.

My (perhaps naive) understanding is that Homebrew installs Borg using a virtual environment... llfuse is therefore located in /usr/local/Cellar/borgbackup/1.1.14_1/libexec/lib/python3.9/site-packages/llfuse.cpython-39-darwin.so.

This makes installing the llfuse dependency after brew install borgbackup more complicated... at least to users who don't know Python... such as me. Is there an easy way to install llfuse in the context of that virtual environment?

@ThomasWaldmann How would you like to handle this in the future? I am working on a Borg series (first episode is out) for the privacy guides... Until now, installing Borg using Homebrew was pretty straight forward. Things are too broken now on macOS for less technology literate viewers.

[ThomasWaldmann]

I don't want to get into caring for distribution/packaging issues.

Especially not for an OS and distribution I do not use and do not know too well.

If brew makes it too complicated, just use pip or the binary from us.

[sunknudsen]

Hey @m3nu, what do you think we should do? I see Vorta depends on borgbackup.

@fxcoudert Are you guys planning on removing the osxfuse cask from Homebrew?

[fxcoudert]

@sunknudsen Casks allow non-open source software, so I don't think it would be removed

[m3nu]

Yes, Vorta depends on Borg, but the Borg formula doesn't depend on osxfuse. After getting some user feedback we made it an optional dependency.

[sunknudsen]

Thanks for the follow-up @m3nu. Have you published instructions on how to install the llfuse dependency after Borg has been installed using brew install vorta?

[m3nu]

llfuse is already contained in Borg's formula. You shouldn't need that.

A note regarding osxfuse will be shown as caveat after installing borgbackup from Homebrew. This is the only way to show optional dependencies AFAIK.

[sunknudsen]

@m3nu

llfuse is already contained in Borg's formula. You shouldn't need that.

Unfortunately, it isn't anymore... see Homebrew/homebrew-core@89785cf.

Homebrew removed llfuse from the recipe a few days ago because FUSE for macOS is no longer open source.

Pleases see comments above.

[m3nu]

I see. Wasn't aware of this change.

[sunknudsen]

@m3nu I am considering maintaining a tap with previous recipe... no sure what is the best way forward.

[sunknudsen]

Hey @ThomasWaldmann and @m3nu, I created a tap that includes a patched version of borgbackup called borgbackup-llfuse. It took me some time to figure out how Homebrew works, but the patched formula appears to be working as expected. I am happy to maintain the patched formula, but it feels like a hack... perhaps it should eventually live within the borgbackup organization.

brew install sunknudsen/tap/borgbackup-llfuse

[sunknudsen]

If you guys are happy with the above workaround, I guess we can close this issue.

[ThomasWaldmann]

@sunknudsen great, thanks for addressing that!

Guess I could create a repo within the borgbackup org for the tap. Preferably only for borgbackup, not for wrappers or other additional software not maintained within the org.

It would of course still needing somebody who cares for it as I do not use macOS / brew myself (besides when doing release testing of borg).

[ThomasWaldmann]

Hmm, one idea:

in your tap, borg and llfuse is in the same formula now. could that also live in 2 formulas, with borg requiring the llfuse one?

i could imagine there might be other users of llfuse suffering from the homebrew issues, so it could be interesting to get it separately even if not using borg. also, in future, we might want to have pyfuse3 (successor of llfuse) also.

[sunknudsen]

@ThomasWaldmann I believe llfuse has to be in the same formula in order to be linked properly in the context of virtualenv.

[sunknudsen]

I would be happy to pitch in to create and maintain (perhaps with @m3nu) a borgbackup/homebrew-tap repo that would only include the borgbackup-llfuse formula.

Full disclosure, I am new to Homebrew and Python so feedback is welcomed.

[ThomasWaldmann]

https://github.com/borgbackup/homebrew-tap there it is!

@sunknudsen i also sent you an invite.

[m3nu]

Happy to also have an eye on this too and keep it working. With Apple's move to ARM we'll see more changes here.

The current tap is roughly what we had before plus a version- and conflict check. Shouldn't break much, but will need docs updates eventually.

[sunknudsen]

@ThomasWaldmann The tap is ready. I just switched my personal Mac from borgbackup to borgbackup-llfuse and confirm things are working as expected.

[ThomasWaldmann]

Guess we need a docs update for 1.1-maint now (install borg on macOS).

I want to release 1.1.15 in december, so it would be cool to have that in before borg (and versioned docs) get released.

[m3nu]

Had some issues when trying the formula today. Will dig deeper tomorrow.

[m3nu]

Works as expected. How did you fix the PyPi issue "Given no hashes to check 151 links ..."?

I added some tests and linting here using GH actions: https://github.com/m3nu/homebrew-tap

[sunknudsen]

Hey @m3nu, the issue was caused by llfuse depending on osxfuse at build time.

[DesertRider]

Hi,

I found this thread while looking why I had this problem after updating borgbackup yesterday... great thread!

I tried to install borgbackup-fuse as instructed on the readme page of homebrew-tap page but it fails with the following:

brew install borgbackup/tap/borgbackup-fuse
==> Tapping borgbackup/tap
Cloning into '/usr/local/Homebrew/Library/Taps/borgbackup/homebrew-tap'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 19 (delta 4), reused 12 (delta 1), pack-reused 0
Unpacking objects: 100% (19/19), done.
Tapped 1 formula (43 files, 33.7KB).
==> Searching for similarly named formulae...
Error: No similarly named formulae found.
Error: No available formula or cask with the name "borgbackup/tap/borgbackup-fuse".
==> Searching for a previously deleted formula (in the last month)...
Error: No previously deleted formula found.

So, first, am I at the right place to discuss the problem?
Did I have to remove the previous installation of borgbackup first?

I'll be glad to help!

[sunknudsen]

Hey @DesertRider, we haven't renamed the formula yet... see borgbackup/homebrew-tap#1. The PR is awaiting review. Please try running brew install borgbackup/tap/borgbackup-llfuse keeping in mind that the formula will be renamed to brew install borgbackup/tap/borgbackup-fuse shortly.

[sunknudsen]

Hey @DesertRider, brew install borgbackup/tap/borgbackup-fuse is live.

[carlosbonilla]

Hey @DesertRider, brew install borgbackup/tap/borgbackup-fuse is live.

@sunknudsen this completely solved the mount issue. Thanks!

Tested on macOS Catalina 10.15.7 and Big Sur 11.01.1

[Gu1nness]

@sunknudsen In order to patch the documentation, what are the ways of installing borgbackup on MacOS?
In particular, are there multiple taps? Other ways?

[sunknudsen]

Hey @Gu1nness, thanks so much for updating the docs.

If you don’t have a Mac, the following episode of the privacy guides shows the process.

This is the part that can likely help with the docs.

https://sunknudsen.com/privacy-guides/how-to-configure-borg-client-on-macos-using-command-line#step-7-install-fuse-for-macos-borg-and-borg-wrapper

In a nutshell, if one doesn’t need to mount backups, one can install Borg using Homebrew’s formula (brew install borgbackup).

If one wishes to mount backups, osxfuse and llfuse are required. Therefore, using Borg’s formula (brew install borgbackup/tap/borgbackup-fuse) is way easier as it handles these dependencies gracefully.

Running brew install borgbackup/tap/borgbackup-fuse before osxfuse is installed will throw an error (see https://github.com/borgbackup/homebrew-tap/blob/master/Formula/borgbackup-fuse.rb#L3-L25).

osxfuse is required to build borgbackup-fuse. Please run brew install --cask osxfuse first.

Therefore, I recommend the following for the docs.

brew install --cask osxfuse
brew install borgbackup/tap/borgbackup-fuse

Let me know if that helps.

[Gu1nness]

Perfect, exactly what I needed to know.
I'll update the docs accordingly :)
(In a few hours, don't have the time for now)

[ThomasWaldmann]

If this should get into 1.1.15, do it soon, please.

[m3nu]

No docs PR yet. Guess I'll do it together with #5622 ?