Can't install on low-space with no work-arounds

[hadess]

$ flatpak install --system flathub io.github.GnomeMpv
Required runtime for io.github.GnomeMpv/x86_64/stable (org.gnome.Platform/x86_64/3.24) is not installed, searching...
Found in several remotes:
1) flathub
2) gnome-x86-tmp
Which do you want to install (0 to abort)? [0-2]: 1
Installing: org.gnome.Platform/x86_64/3.24 from flathub
error: min-free-space-percent '3%' would be exceeded, 5.1 GB available

It doesn't tell me where it's trying to install, why it's trying to install there (my / is 50% full with about 25GB left), why that percentage of free space must be kept (if it's what it says, because that looks like a configuration option), or how to fix it or work-around it.

Using flatpak-0.9.7-5.fc27.x86_64

[alexlarsson]

It writes the temporary download to your homedir before applying to the system repo, so I guess it is for that. However, I've never seen this error before. Seems like a new feature in ostree 2017.8.

@cgwalters What was the reasoning about the 3% default. I'm wondering if I should change that in flatpak.

Anyway, @hadess you should be able to change this by setting something like:

sudo ostree --repo=/var/lib/flatpak/repo/ config set core.min-free-space-percent 1

(Didn't actually test this)

[alexlarsson]

@hadess Also, do you have any mountpoint with 5GB free, maybe its measuring it wrong?

[hadess]

Could be my home directory:

/dev/sda5       179G  165G  4.7G  98% /home

error: min-free-space-percent '3%' would be exceeded, 5.0 GB available

My home directory is always chock-full, so I guess this is going to be a recurring problem. The config set command above didn't solve the problem.

[alexlarsson]

Hmm, we do a download to a temporary repo that is a "child" of the system repo, and that one is in the homedir. I was assuming it would inherit the free-space property from the parent, but apparently not.

as a workaround, sudo flatpak install will avoid writing to the home dir, but that is not really a proper fix.

[hadess]

as a workaround, sudo flatpak install will avoid writing to the home dir, but that is not really a proper fix.

I figured that one out, but I think it's the installation of the runtime that triggers it, so didn't want to break my test case just yet :)

[alexlarsson]

I'm not sure what the proper fix is here. I mean, we could automatically set the "child repo" to 0% min free. But is that generally a good idea? Filling up the users homedir is not ideal either.

Another alternative is to put the temporary repo under /var/tmp, hopeing there is more space there. But thats kinda arbitrary too, and may run into similar issues...

[cgwalters]

3% was fairly made-up, just took inspiration from the traditional extfs 5%.

But here it seems like the feature is working correctly, we avoided filling up your home directory, right?

[cgwalters]

For reference this is from ostreedev/ostree#987

[cgwalters]

Now if this issue is about explaining things better, we could make a new error domain for this, and flatpak could probably use it to render more useful text.

[hadess]

But here it seems like the feature is working correctly, we avoided filling up your home directory, right?

Seeing as the final destination of the download would have been the root mount (which has about 25GB available), and that I couldn't do the installation, it really didn't work out great, no.

[cgwalters]

Right, but the only thing that would make that work is something like what Alex suggested:

Another alternative is to put the temporary repo under /var/tmp, hopeing there is more space there.
But thats kinda arbitrary too, and may run into similar issues...

[hadess]

FWIW, gnome-software seems to silently fail with the same error when trying to install.

IMO, 3% is a silly number, just as 1% reserved for root on ext4 systems is silly nowadays. 3% of my /home partition (on an SSD) is 5 gigs. I don't think that the runtime actually uses anywhere near that, and this would be even worse with the kind of spinning disk sizes as well.

ostree giving itself leeway to not get to a point where it clogs up the system is admirable, but I don't think that this decision is being taken at the right level. Especially when it means I can't even update software now, let alone install new one.

[alexlarsson]

I agree that percentages are not really a great measure anymore. 3% of my spinning rust drive is 27 gig.

[alexlarsson]

@hadess Can you see if that "fixes" the workaround for you?

[cgwalters]

One thing that influenced my thoughts on picking a number here is that ostree is designed for code, not data (as distinct from the ext4 number which is for both). And data is usually much larger than code. It seems rare to me that one would actually want to fill up a storage device with code.

[hellpe]

I think I'm having the same issue. I've got 19,9 GB available on my 1 TB hard drive (with is more than 3% isn't it?), and I can't do Flatpak-related tasks anymore. What is exactly the workaround you're mentioning in our last comment, @alexlarsson? Can I perform it using my current version of ostree (2017.10-1on Arch Linux)?

[nekohayo]

Got the same issue, and yeah, warn me if you must, but don't force fail on such a simple heuristic.

Unfortunately all that I have on hand is the flatpak version provided with Fedora 26 so that's the issue I'm seeing right now, I'll have to wait for updates to trickle down to be able to retest this.

[cgwalters]

The change alex pushed to master: 706d138
should allow working around this client side.

Part of this issue would also be addressed if flatpak did pulls in /var/tmp as alex suggested too; I think that's more clearly right; besides the free space issue, it would also eventually allow libostree to use filesystem-native clones via copy_file_range if available. Maybe something like create ${XDG_RUNTIME_DIR}/flatpak-temp-repo as a symlink to it so it's discoverable but still randomly named? (Too bad there's not a well-defined per-user /var/tmp dir automatically managed via logind or whatever)

Now...we can definitely make changes in libostree for this; but it's not clear what. A notable wrinkle here is that while it's clear there's no security issues here with allowing users to fill up their own homedirs for flatpak install --user, I'd say we should have some policy around system-wide installs.

[Jehan]

Hi!

I've got the same issue while I have 604GB available in /home/ and I try to update with --user! Since I notice that the rest of my system only has 532MB remaining, I guess that's the issue. Yet it's not a system update so the home space should be enough.

$ flatpak --user update
Looking for updates...
Updating: org.gnome.Platform/x86_64/3.24 from gnome
Warning: Failed to update org.gnome.Platform/x86_64/3.24: min-free-space-percent '3%' would be exceeded, 558.4 MB available
Updating: org.gnome.Platform.Locale/x86_64/3.24 from gnome
Warning: Failed to update org.gnome.Platform.Locale/x86_64/3.24: min-free-space-percent '3%' would be exceeded, 558.4 MB available
Updating: org.pitivi.Pitivi/x86_64/stable from org.pitivi.Pitivi-origin
Error: Failed to update org.pitivi.Pitivi/x86_64/stable: min-free-space-percent '3%' would be exceeded, 558.4 MB available
error: One or more operations failed
operations failed
$ df -h
Filesystem                          Size  Used Avail Use% Mounted on
devtmpfs                            3.9G     0  3.9G   0% /dev
tmpfs                               3.9G  104K  3.9G   1% /dev/shm
tmpfs                               3.9G  1.6M  3.9G   1% /run
tmpfs                               3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/fedora_darkmarmot-root   50G   47G  532M  99% /
tmpfs                               3.9G  688K  3.9G   1% /tmp
/dev/sda1                           976M  178M  731M  20% /boot
/dev/mapper/fedora_darkmarmot-home  859G  212G  604G  26% /home
tmpfs                               787M   16K  787M   1% /run/user/42
tmpfs                               787M   40K  787M   1% /run/user/1000

[alexlarsson]

This seems like a pure miscalculation of the percentage. --user update should be a regular pull in a directory inside /home, with no other complications, and 3% of 859G is 26G, so it should be nowhere near 97% full.

@cgwalters Have you seen something like this?

What filesystem type is /home?

@Jehan Can you try this:

strace -o log -f -yy flatpak --user update org.gnome.Platform/x86_64/3.24

And then put the log file somewhere we can look at it? Then we should be able to see what diskspace is reported by the statfs calls, and where.

[Jehan]

Unfortunately it doesn't do it anymore. I guess you don't want this strace anymore?

[alexlarsson]

I disabled the min space check by default here:
5578033

[Arthur-D]

I hit this issue with Flatpak 0.10.1 installed from your PPA. Did it not make it for the release?

[hellpe]

I'm also having this issue again with 0.10.1 (Arch Linux). Here is what happens when I'm trying to install a flatpak with GNOME Software:

This time though, using the workaround that was suggested to me on IRC last September (ostree --repo=/home/hellpe/.local/share/flatpak/repo/ config set core.min-free-space-percent 0) didn't work.

[trappedinspacetime]

You must issue that command to your local repo ostree --repo=/home/hellpe/somelocaldir/flatpak-app/my-repo/ config set core.min-free-space-percent 0